From erik.m.bray at gmail.com Mon Oct 1 10:51:11 2012 From: erik.m.bray at gmail.com (Erik Bray) Date: Mon, 1 Oct 2012 10:51:11 -0400 Subject: [IPython-dev] Weird PYTHONPATH, etc. issue In-Reply-To: References: Message-ID: On Thu, Sep 27, 2012 at 3:12 PM, David Warde-Farley wrote: > On Thu, Sep 27, 2012 at 8:59 AM, Erik Bray wrote: > >> I don't really see this so much as a setuptools problem though. Your >> sysadmin shouldn't be using PYTHONPATH for this. I've had to battle >> this a bit at my own institution. The problem with using PYTHONPATH >> to add site-specific site-packages directories are many-fold. For >> one, if the user then messes with their PYTHONPATH they suddenly can't >> `import numpy` anymore and come complaining that numpy broke or >> something like that. PYTHONPATH should be there as an easy way for >> the user to optionally add additional paths to the front of their >> sys.path. So when it's used like this it basically takes away your >> freedom to customize anything on your path at all since the contents >> of PYTHONPATH come first. Sure you can always add your stuff to the >> front of PYTHONPATH too, but a lot of users get confused by that. > > Prepending to an environment variable to override things is a pretty > standard UNIXy thing to do. Same thing appears in Java with CLASSPATH, > etc., and it pretty much mirrors PATH in that respect. I guess it > could confuse novices, you're right. I agree that it's a pretty standard UNIXy thing to do; and that fact was tugging on the back of my mind as I wrote that. Python, however, really does not need to rely on PYTHONPATH as much as Java, for example, relies on CLASSPATH. And this is a Good Thing. Python is designed so that lots of things can be added to its import path (through sitecustomize.py, usercustomize.py, and other mechanisms) while keeping PYTHONPATH completely clean, allowing it to be used easily as a mechanism for run-time user overrides *without* requiring the user to take great care to keep all the correct items in it, and carefully restore in when they're done with this. That's one of the great things about virtualenv--it often surprises people when I show them virtualenv for the first time that it doesn't put anything on their PYTHONPATH (although it does modify $PATH). Relying on PYTHONPATH makes virtualenv less useful because you can't install something into the virtualenv (a different version of Numpy for example) and have it override the Numpy imported via the PYTHONPATH entries. But I digress. >> What the sysadmin *should* be doing is adding a sitecustomize.py >> somewhere on the standard sys.path (in lib/pythonX.Y/site-packages for >> example) and adding any custom paths there, while leaving your >> PYTHONPATH alone. As mentioned elsewhere in this thread >> usercustomize.py can also be used. > > Are you sure this would solve the issue? If all of the stuff in all of > the site-packages directories is processed before PYTHONPATH is even > touched, then I guess it might. > > David Generally speaking, yes. Though I would still believe it if, as Fernando wrote, setuptools is doing something that even messes up/overrides PYTHONPATH. I'm just not sure what that would be as I've never run into any problems like that. But I'm not saying they don't exist, or even denying that they're likely to exist :) Erik From brad.froehle at gmail.com Mon Oct 1 22:05:19 2012 From: brad.froehle at gmail.com (Bradley M. Froehle) Date: Mon, 1 Oct 2012 19:05:19 -0700 Subject: [IPython-dev] IPEP-4: Python 3 Compatibility Message-ID: Hi, I'd like to introduce "IPEP-4: Python 3 Compatibility", the fourth IPython Enhancement Proposal. https://github.com/ipython/ipython/wiki/IPython-Enhancement-Proposals-(IPEPs) https://github.com/ipython/ipython/wiki/IPEP-4:-Python-3-Compatibility As you likely know, currently running ``python3 ipython.py`` does not work out of the box, as the source code we ship is only natively compatible with Python 2.6 and 2.7. Our excellent support for Python 3.2 and 3.3 relies on first running the source code through 2to3 (i.e., ``python3 setup,py build``). This proposal suggests a number of modifications to the IPython source to allow the code to be run in Python 2.6, 2.7 and 3.3 without the use of 2to3. Python 3.2 compatibility would be maintained through continued use of 2to3. If you are interested in fulfilling the "instant running" promise ``python3.3 ipython.py`` or detest the (invasive?) required source code changes and think we should defer this project until a later date, here is what you should do: 1. Read the draft IPEP: https://github.com/ipython/ipython/wiki/IPEP-4:-Python-3-Compatibility 2. Discuss the IPEP in this email thread and/or in the Github issue queue: https://github.com/ipython/ipython/issues/2440 3. Help us edit the IPEP as the discussion moves along. Cheers, Brad From satra at mit.edu Mon Oct 1 22:50:12 2012 From: satra at mit.edu (Satrajit Ghosh) Date: Mon, 1 Oct 2012 22:50:12 -0400 Subject: [IPython-dev] how to deal with really large cell output Message-ID: hi all, i ran into a scenario recently, where the stdout from a command resulted in a notebook file that was 23MB large and it brought the browser to a halt and the same when loading the notebook. are there ways around this? cheers, satra -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Tue Oct 2 04:00:13 2012 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Tue, 2 Oct 2012 10:00:13 +0200 Subject: [IPython-dev] how to deal with really large cell output In-Reply-To: References: Message-ID: We plan on adding progressive load/ unload of un necessary part, cell by cell. This need refactor, but should fix most of the issues of huge notebooks loading. Sorry for the inconvenience. Le 2 oct. 2012 04:50, "Satrajit Ghosh" a ?crit : > hi all, > > i ran into a scenario recently, where the stdout from a command resulted > in a notebook file that was 23MB large and it brought the browser to a halt > and the same when loading the notebook. are there ways around this? > > cheers, > > satra > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From satra at mit.edu Tue Oct 2 08:00:56 2012 From: satra at mit.edu (Satrajit Ghosh) Date: Tue, 2 Oct 2012 08:00:56 -0400 Subject: [IPython-dev] how to deal with really large cell output In-Reply-To: References: Message-ID: thanks matthias. for the time being i loaded the notebook in emacs and removed the output from the cell. cheers, satra On Tue, Oct 2, 2012 at 4:00 AM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > We plan on adding progressive load/ unload of un necessary part, cell by > cell. This need refactor, but should fix most of the issues of huge > notebooks loading. > > Sorry for the inconvenience. > Le 2 oct. 2012 04:50, "Satrajit Ghosh" a ?crit : > >> hi all, >> >> i ran into a scenario recently, where the stdout from a command resulted >> in a notebook file that was 23MB large and it brought the browser to a halt >> and the same when loading the notebook. are there ways around this? >> >> cheers, >> >> satra >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans_meine at gmx.net Tue Oct 2 08:40:58 2012 From: hans_meine at gmx.net (Hans Meine) Date: Tue, 02 Oct 2012 14:40:58 +0200 Subject: [IPython-dev] how to deal with really large cell output In-Reply-To: References: Message-ID: <4384350.bmmOWs1VgD@hmeine-pc> Am Dienstag, 2. Oktober 2012, 08:00:56 schrieb Satrajit Ghosh: > thanks matthias. for the time being i loaded the notebook in emacs and > removed the output from the cell. Have you tried the Emacs IPython notebook? http://tkf.github.com/emacs-ipython-notebook/ It was incredibly straight-forward to setup and use, and finally offers *both* the fabulous IPython notebook *and* your favorite code editor. Just sayin'. ;-) Best regards, Hans From satra at mit.edu Tue Oct 2 11:39:13 2012 From: satra at mit.edu (Satrajit Ghosh) Date: Tue, 2 Oct 2012 11:39:13 -0400 Subject: [IPython-dev] traitlets instance metadata Message-ID: hi devs, does the traitlets implementation allow metadata modification on a per instance basis? cheers, satra -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Tue Oct 2 12:21:00 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 2 Oct 2012 09:21:00 -0700 Subject: [IPython-dev] how to deal with really large cell output In-Reply-To: References: Message-ID: It is difficult for any environment to handle that about of stdout - not just the notebook. So my first answer is don't do that. But I realize that these things to happen occasionally (sometimes you don't know how big something really is - it is pretty easy for this to happen accidentally). What we should do is put a limit on how much gets sent back to the browser and then have an API/UI for getting more if you want. On Mon, Oct 1, 2012 at 7:50 PM, Satrajit Ghosh wrote: > hi all, > > i ran into a scenario recently, where the stdout from a command resulted in > a notebook file that was 23MB large and it brought the browser to a halt and > the same when loading the notebook. are there ways around this? > > cheers, > > satra > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From ellisonbg at gmail.com Tue Oct 2 12:21:00 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 2 Oct 2012 09:21:00 -0700 Subject: [IPython-dev] how to deal with really large cell output In-Reply-To: References: Message-ID: It is difficult for any environment to handle that about of stdout - not just the notebook. So my first answer is don't do that. But I realize that these things to happen occasionally (sometimes you don't know how big something really is - it is pretty easy for this to happen accidentally). What we should do is put a limit on how much gets sent back to the browser and then have an API/UI for getting more if you want. On Mon, Oct 1, 2012 at 7:50 PM, Satrajit Ghosh wrote: > hi all, > > i ran into a scenario recently, where the stdout from a command resulted in > a notebook file that was 23MB large and it brought the browser to a halt and > the same when loading the notebook. are there ways around this? > > cheers, > > satra > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From asmeurer at gmail.com Tue Oct 2 14:18:17 2012 From: asmeurer at gmail.com (Aaron Meurer) Date: Tue, 2 Oct 2012 12:18:17 -0600 Subject: [IPython-dev] how to deal with really large cell output In-Reply-To: <4384350.bmmOWs1VgD@hmeine-pc> References: <4384350.bmmOWs1VgD@hmeine-pc> Message-ID: <6564895197062025063@unknownmsgid> On Oct 2, 2012, at 6:41 AM, Hans Meine wrote: > Am Dienstag, 2. Oktober 2012, 08:00:56 schrieb Satrajit Ghosh: >> thanks matthias. for the time being i loaded the notebook in emacs and >> removed the output from the cell. > > Have you tried the Emacs IPython notebook? > http://tkf.github.com/emacs-ipython-notebook/ > > It was incredibly straight-forward to setup and use, and finally offers *both* > the fabulous IPython notebook *and* your favorite code editor. I'm actually curious how the emacs notebook would have handled that file. Emacs can handle enormous files with a breeze because it is able to load the file dynamically (though certain extensions or lines in .emacs can negate this). I've used it to edit files that are a gigabyte large, which crashed my best GUI editor. So if it was just an issue of JavaScript and the browser, perhaps emacs could have fared better. Aaron Meurer > > Just sayin'. ;-) > > Best regards, > Hans > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From dpquigl at davequigley.com Tue Oct 2 15:43:44 2012 From: dpquigl at davequigley.com (David Quigley) Date: Tue, 02 Oct 2012 15:43:44 -0400 Subject: [IPython-dev] =?utf-8?q?Anyone_done_anything_like_this_before=3F?= Message-ID: I want to use ipython as a shell which will receive commands and then task a process to do things. The trick here is that I want to start the process from within ipython and monitor its stdin/stdout for commands. It is a message passing interface so events will come in async and I need the shell to be able to handle them. Is there something similar to this out there somewhere that I can look at for inspiration? Is this even possible? Dave From aron at ahmadia.net Tue Oct 2 15:47:00 2012 From: aron at ahmadia.net (Aron Ahmadia) Date: Tue, 2 Oct 2012 22:47:00 +0300 Subject: [IPython-dev] Anyone done anything like this before? In-Reply-To: References: Message-ID: Absolutely, you want the subprocess module, and in particular, the subprocess.communicate() method. http://docs.python.org/library/subprocess.html This is core Python functionality, but it wil work from IPython as well. A On Tue, Oct 2, 2012 at 10:43 PM, David Quigley wrote: > I want to use ipython as a shell which will receive commands and then > task a process to do things. The trick here is that I want to start the > process from within ipython and monitor its stdin/stdout for commands. > It is a message passing interface so events will come in async and I > need the shell to be able to handle them. Is there something similar to > this out there somewhere that I can look at for inspiration? Is this > even possible? > > Dave > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dpquigl at davequigley.com Tue Oct 2 15:51:02 2012 From: dpquigl at davequigley.com (David Quigley) Date: Tue, 02 Oct 2012 15:51:02 -0400 Subject: [IPython-dev] =?utf-8?q?Anyone_done_anything_like_this_before=3F?= In-Reply-To: References: Message-ID: On 10/02/2012 15:47, Aron Ahmadia wrote: > Absolutely, you want the subprocess module, and in particular, the > subprocess.communicate() method. > > http://docs.python.org/library/subprocess.html [3] > > This is core Python functionality, but it wil work from IPython as > well. > > A > > On Tue, Oct 2, 2012 at 10:43 PM, David Quigley > wrote: > >> I want to use ipython as a shell which will receive commands and >> then >> task a process to do things. The trick here is that I want to start >> the >> process from within ipython and monitor its stdin/stdout for >> commands. >> It is a message passing interface so events will come in async and I >> need the shell to be able to handle them. Is there something similar >> to >> this out there somewhere that I can look at for inspiration? Is this >> even possible? >> >> Dave >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev [2] > > > > Links: > ------ > [1] http://mail.scipy.org/mailman/listinfo/ipython-dev > [2] http://docs.python.org/library/subprocess.html > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev I've started down that path and I'm hoping I can select on the pipes for STDIN and STDOUT. I'm just wondering how this fits in with ipython's event model. Maybe its my lack of knowledge of python but our custom shell we already have written uses an io event loop to just listen for messages on the two pipes and process and send over them as needed. I guess the better way of phrasing the question would be, do I have to worry about decoupling an IO loop for the subprocess from the io loop for ipython? From aron at ahmadia.net Tue Oct 2 15:57:06 2012 From: aron at ahmadia.net (Aron Ahmadia) Date: Tue, 2 Oct 2012 22:57:06 +0300 Subject: [IPython-dev] Anyone done anything like this before? In-Reply-To: References: Message-ID: I've started down that path and I'm hoping I can select on the pipes > for STDIN and STDOUT. I'm just wondering how this fits in with ipython's > event model. Maybe its my lack of knowledge of python but our custom > shell we already have written uses an io event loop to just listen for > messages on the two pipes and process and send over them as needed. I > guess the better way of phrasing the question would be, do I have to > worry about decoupling an IO loop for the subprocess from the io loop > for ipython? > > Fair enough, this goes deeper into the IPython internals than I'm familiar with, so you might need to wait for one of the real experts to reply :) A -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Tue Oct 2 16:21:01 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 2 Oct 2012 13:21:01 -0700 Subject: [IPython-dev] Policy for closing pull requests In-Reply-To: References: Message-ID: https://github.com/ipython/ipython/wiki/Policy:-Closing-pull-requests Please make edits if I have missed anything. On Mon, Aug 13, 2012 at 8:29 PM, Fernando Perez wrote: > On Mon, Aug 13, 2012 at 8:14 PM, Brian Granger wrote: >> On Sun, Aug 12, 2012 at 11:36 PM, Fernando Perez wrote: > >>>> * What do we do about PRs that sit for a month waiting for review? >>> >>> We flog ourselves with a nail-studded whip and we review them :) >> >> I went back and forth of this one. I mostly agree with you. But our >> busy schedules and the large number of PRs definitely makes it a >> challenge to review everything promptly. > > I know, this one is terrible. I really hope we never reach that > point, and that's why at least I think we can ask the wider list to > pitch in with review if we're falling behind badly. Because if it's > only up to the core committers to keep the queue moving, we'll hit > this bad scenario way too often. > >>> I want IPython to be a project where, if we have *one* policy, it's >>> that intelligent discussion comes *always* before policy. >> >> Yes, but part of my goal is that we can come up with a guidelines for >> closing PRs that are simple, and don't require long discussions. If I > > Oh, certainly: I think having guidelines like this *is* useful, as it > lets us make frequent decisions fluidly. This is similar to how we > have other good dev guidelines that make the process easier for > everyone. I just don't want to go too far in the direction of policy > overriding the notion that we're open to discussion *when discussion > is really necessary*. But if the guidelines are sensible, in the vast > majority of cases they can just be applied/followed, no discussion is > necessary and they just help keep the gears in motion. > >> or someone else adheres to these guidelines are you OK with PRs being >> closed without first running it by all interested parties? To >> summarize, here are the main points: >> >> * PRs that haven't been reviewed for a month should prompt us to kick >> each other into gear. We should ping each other, the list and >> specific people if necessary to get the review going. The PR remains >> open and we apologize to the author for our slowness. >> * PRs that have been reviewed, but that sit for a month waiting for >> code get closed. >> * PRs that spawn larger discussions that are not resolved after a >> month get closed. >> * The person who closes an issue must open an issue that links to the > > you mean 'who closes a PR' > >> PR and tell the author that we welcome them to reopen the PR when they >> start to work on it again. If appropriate we can even give details of >> the work that needs to be done. But the tone must be one of >> encouragement. We can even site this guideline and our desire to keep >> PRs moving. > > Yup, I think this is a *very* sensible set of guidelines, and I bet > you we'll be able to follow them more or less automatically in most > cases. > > And furthermore, it means we can have a little script that runs as a > cron job somewhere and updates a little dashboard on the future wiki > showing perhaps the state of the PR queue as a green/yellow/red grid: > red ones are over one month of inactivity (hence candidates to go and > close with the above process), yellow ones have say > 2 weeks of > inactivity, and those with activity in the last two weeks are green. > > I'd really like to have a birds-eye view of the PR queue like that :) > If in practice we find that it's too easy to 'refresh' a PR to green > status because of an innocent comment that doesn't really add > anything, we could tweak the above to be time since last pushed commit > instead of time since last comment... Just an idea... > > In any case, that was just me thinking about workflow tools. But back > to the basic point of this discussion, I'm +1 on the above summary. > > Thanks for taking this on! It will help us to organize the flow better. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From takowl at gmail.com Tue Oct 2 16:42:08 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 2 Oct 2012 21:42:08 +0100 Subject: [IPython-dev] Anyone done anything like this before? In-Reply-To: References: Message-ID: On 2 October 2012 20:47, Aron Ahmadia wrote: > Absolutely, you want the subprocess module, and in particular, the > subprocess.communicate() method. > > http://docs.python.org/library/subprocess.html I'm not sure that that is the best thing here. The docs for communicate() say "...Wait for process to terminate...", whereas it sounds like David wants to repeatedly communicate with a process. I'm not entirely sure from the description, but I'd point you to pexpect, which is designed to wait for particular output patterns from a process and respond with input. http://www.noah.org/wiki/pexpect#Description_of_Pexpect Thomas From dpquigl at davequigley.com Tue Oct 2 16:45:46 2012 From: dpquigl at davequigley.com (David Quigley) Date: Tue, 02 Oct 2012 16:45:46 -0400 Subject: [IPython-dev] =?utf-8?q?Anyone_done_anything_like_this_before=3F?= In-Reply-To: References: Message-ID: On 10/02/2012 16:42, Thomas Kluyver wrote: > On 2 October 2012 20:47, Aron Ahmadia wrote: >> Absolutely, you want the subprocess module, and in particular, the >> subprocess.communicate() method. >> >> http://docs.python.org/library/subprocess.html > > I'm not sure that that is the best thing here. The docs for > communicate() say "...Wait for process to terminate...", whereas it > sounds like David wants to repeatedly communicate with a process. > > I'm not entirely sure from the description, but I'd point you to > pexpect, which is designed to wait for particular output patterns > from > a process and respond with input. > > http://www.noah.org/wiki/pexpect#Description_of_Pexpect > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev I ran into that problem before. If you don't want to wait for it to terminate you can use the IO pipes directly instead. Select on the STDIN or STDOUT that you get from the popen call and use those to communicate with the process. From takowl at gmail.com Tue Oct 2 17:00:32 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 2 Oct 2012 22:00:32 +0100 Subject: [IPython-dev] Anyone done anything like this before? In-Reply-To: References: Message-ID: On 2 October 2012 21:45, David Quigley wrote: > I ran into that problem before. If you don't want to wait for it to > terminate you can use the IO pipes directly instead. Select on the STDIN > or STDOUT that you get from the popen call and use those to communicate > with the process. The subprocess docs caution against using stdin/stdout directly, but the reasoning is something to do with buffers and deadlocks that I don't understand. If you're happy with it, go ahead. I'm not quite sure what you're asking about IPython. The basic terminal IPython doesn't have an event loop: it just calls the raw_input() function to get user commands, i.e. it's blocking. The two-process model (for the Qt console, the notebook, etc.), uses the event loop in pyzmq, which is based on the Tornado event loop. Details here: http://zeromq.github.com/pyzmq/eventloop.html Thomas From takowl at gmail.com Tue Oct 2 17:45:34 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 2 Oct 2012 22:45:34 +0100 Subject: [IPython-dev] Extension loading In-Reply-To: References: Message-ID: On 29 September 2012 16:42, Thomas Kluyver wrote: > Some of our default extensions contain a global flag to prevent the > extension being loaded more than once. This is rather ugly, and fixes > us to the singleton model of one IPython shell per process. I'd like > to clear that up before more third parties copy the pattern. I've now started this in PR #2462 https://github.com/ipython/ipython/pull/2462 Thanks, Thomas From jason-sage at creativetrax.com Tue Oct 2 18:34:10 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Tue, 02 Oct 2012 17:34:10 -0500 Subject: [IPython-dev] Anyone done anything like this before? In-Reply-To: References: Message-ID: <506B6BE2.6090504@creativetrax.com> On 10/2/12 3:42 PM, Thomas Kluyver wrote: > On 2 October 2012 20:47, Aron Ahmadia wrote: >> Absolutely, you want the subprocess module, and in particular, the >> subprocess.communicate() method. >> >> http://docs.python.org/library/subprocess.html > > I'm not sure that that is the best thing here. The docs for > communicate() say "...Wait for process to terminate...", whereas it > sounds like David wants to repeatedly communicate with a process. That note in the communicate docs: http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate indicates you should use Popen with a stdin/stdout pipe to be able to do what you want. Thanks, Jason From satra at mit.edu Wed Oct 3 14:17:44 2012 From: satra at mit.edu (Satrajit Ghosh) Date: Wed, 3 Oct 2012 14:17:44 -0400 Subject: [IPython-dev] multiprocessing from within notebook crashes kernel In-Reply-To: References: Message-ID: hi min, just getting back to this error. turns out that this error results from an interaction between our library's logging mechanism (using python logging module), the ipython kernel (happens with notebook and console but not with ipython on terminal) and multiprocessing. i don't get the same error running the same set of commands in the ipython terminal. i suspect it has to do with how logging output is trapped and conveyed in the multiprocessing context back to the browser. cheers, satra On Sun, Sep 9, 2012 at 2:22 PM, MinRK wrote: > What version of pyzmq? If it's less than 2.2.0.1, try upgrading. > -MinRK > > On Sun, Sep 9, 2012 at 6:35 AM, Satrajit Ghosh wrote: > >> hi all, >> >> we are using a workflow library that internally uses multiprocessing to >> distribute jobs locally. when we run this from the notebook, it crashes the >> kernel. >> >> any thoughts as to why this might happen. >> >> here is the code from the library: >> >> >> https://github.com/nipy/nipype/blob/master/nipype/pipeline/plugins/multiproc.py >> >> cheers, >> >> satra >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Wed Oct 3 14:26:27 2012 From: benjaminrk at gmail.com (MinRK) Date: Wed, 3 Oct 2012 11:26:27 -0700 Subject: [IPython-dev] multiprocessing from within notebook crashes kernel In-Reply-To: References: Message-ID: On Wed, Oct 3, 2012 at 11:17 AM, Satrajit Ghosh wrote: > hi min, > > just getting back to this error. turns out that this error results from an > interaction between our library's logging mechanism (using python logging > module), the ipython kernel (happens with notebook and console but not with > ipython on terminal) and multiprocessing. i don't get the same error > running the same set of commands in the ipython terminal. i suspect it has > to do with how logging output is trapped and conveyed in the > multiprocessing context back to the browser. > Anything that tries to write to sys.stdout/err from a fork will definitely crash the process: https://github.com/ipython/ipython/issues/2438 > > cheers, > > satra > > > On Sun, Sep 9, 2012 at 2:22 PM, MinRK wrote: > >> What version of pyzmq? If it's less than 2.2.0.1, try upgrading. >> -MinRK >> >> On Sun, Sep 9, 2012 at 6:35 AM, Satrajit Ghosh wrote: >> >>> hi all, >>> >>> we are using a workflow library that internally uses multiprocessing to >>> distribute jobs locally. when we run this from the notebook, it crashes the >>> kernel. >>> >>> any thoughts as to why this might happen. >>> >>> here is the code from the library: >>> >>> >>> https://github.com/nipy/nipype/blob/master/nipype/pipeline/plugins/multiproc.py >>> >>> cheers, >>> >>> satra >>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From satra at mit.edu Wed Oct 3 16:56:47 2012 From: satra at mit.edu (Satrajit Ghosh) Date: Wed, 3 Oct 2012 16:56:47 -0400 Subject: [IPython-dev] multiprocessing from within notebook crashes kernel In-Reply-To: References: Message-ID: thanks min. that's exactly it. cheers, satra On Wed, Oct 3, 2012 at 2:26 PM, MinRK wrote: > > > On Wed, Oct 3, 2012 at 11:17 AM, Satrajit Ghosh wrote: > >> hi min, >> >> just getting back to this error. turns out that this error results from >> an interaction between our library's logging mechanism (using python >> logging module), the ipython kernel (happens with notebook and console but >> not with ipython on terminal) and multiprocessing. i don't get the same >> error running the same set of commands in the ipython terminal. i suspect >> it has to do with how logging output is trapped and conveyed in the >> multiprocessing context back to the browser. >> > > Anything that tries to write to sys.stdout/err from a fork will definitely > crash the process: https://github.com/ipython/ipython/issues/2438 > > >> >> cheers, >> >> satra >> >> >> On Sun, Sep 9, 2012 at 2:22 PM, MinRK wrote: >> >>> What version of pyzmq? If it's less than 2.2.0.1, try upgrading. >>> -MinRK >>> >>> On Sun, Sep 9, 2012 at 6:35 AM, Satrajit Ghosh wrote: >>> >>>> hi all, >>>> >>>> we are using a workflow library that internally uses multiprocessing to >>>> distribute jobs locally. when we run this from the notebook, it crashes the >>>> kernel. >>>> >>>> any thoughts as to why this might happen. >>>> >>>> here is the code from the library: >>>> >>>> >>>> https://github.com/nipy/nipype/blob/master/nipype/pipeline/plugins/multiproc.py >>>> >>>> cheers, >>>> >>>> satra >>>> >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Thu Oct 4 13:16:22 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 4 Oct 2012 10:16:22 -0700 Subject: [IPython-dev] SWC using IPython Notebook Message-ID: HI, Software Carpentry just announced that they are going to transition all of their materials to the IPython Notebook. This is a huge victory for us, congrats everyone! Matthias - an important part of the story for them was having notebooks viewable online using nbviewer - again, great work on that. Cheers, Brian -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From takowl at gmail.com Thu Oct 4 13:29:12 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 4 Oct 2012 18:29:12 +0100 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: On 4 October 2012 18:16, Brian Granger wrote: > Software Carpentry just announced that they are going to transition > all of their materials to the IPython Notebook. This is a huge > victory for us, congrats everyone! Excellent! Congratulations to everyone who's worked on it. For reference, here's the blog post: http://software-carpentry.org/2012/10/transitioning-to-the-ipython-notebook/ Best wishes, Thomas From ellisonbg at gmail.com Thu Oct 4 13:40:41 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 4 Oct 2012 10:40:41 -0700 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: Doh, forgot to add the link, thanks Thomas! On Thu, Oct 4, 2012 at 10:29 AM, Thomas Kluyver wrote: > On 4 October 2012 18:16, Brian Granger wrote: >> Software Carpentry just announced that they are going to transition >> all of their materials to the IPython Notebook. This is a huge >> victory for us, congrats everyone! > > Excellent! Congratulations to everyone who's worked on it. > > For reference, here's the blog post: > http://software-carpentry.org/2012/10/transitioning-to-the-ipython-notebook/ > > Best wishes, > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From takowl at gmail.com Thu Oct 4 13:46:02 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 4 Oct 2012 18:46:02 +0100 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: On 4 October 2012 18:40, Brian Granger wrote: > Doh, forgot to add the link, thanks Thomas! You're welcome. Have you already left a comment on the blog post? If not, I'll write one like "Thanks on behalf of the IPython team, don't hesitate to let us know if you run into any problems." Thomas From ellisonbg at gmail.com Thu Oct 4 13:49:16 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 4 Oct 2012 10:49:16 -0700 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: I haven't, but will do that now. But I think all of us who want should leave comments to show our excitement and support. On Thu, Oct 4, 2012 at 10:46 AM, Thomas Kluyver wrote: > On 4 October 2012 18:40, Brian Granger wrote: >> Doh, forgot to add the link, thanks Thomas! > > You're welcome. Have you already left a comment on the blog post? If > not, I'll write one like "Thanks on behalf of the IPython team, don't > hesitate to let us know if you run into any problems." > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From jiffyclub at gmail.com Thu Oct 4 17:28:34 2012 From: jiffyclub at gmail.com (Matt Davis) Date: Thu, 4 Oct 2012 17:28:34 -0400 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: Hi all, Thank you for putting together this great tool! (I'll admit to being pretty blas? on IPython until you came out with the Notebook.) We've also got a PyCon proposal in on the topic of teaching with Notebooks. There is already a lot of communication between SWC and the IPython devs so none of these requests are exactly news, but here are some things we'd like for the teaching side: - Integration with pythontutor.com - A way for an instructor to make a notebook then strip out code for distribution to students (so they can see our annotations but have to type their own code). - A way to put hints into Markdown cells, e.g. collapsed by default but expandable if students want help. I do pretty much all my in-person teaching with live typing but the slide show abilities also look quite cool and I'm sure we'll put those to use too! Thanks again for all your hard work! Hopefully one of these days I'll have some time to give back. Best, Matt Davis On Thu, Oct 4, 2012 at 1:16 PM, Brian Granger wrote: > HI, > > Software Carpentry just announced that they are going to transition > all of their materials to the IPython Notebook. This is a huge > victory for us, congrats everyone! > > Matthias - an important part of the story for them was having > notebooks viewable online using nbviewer - again, great work on that. > > Cheers, > > Brian > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Thu Oct 4 17:41:16 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 4 Oct 2012 14:41:16 -0700 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: Hi, Sent from my iPad On Oct 4, 2012, at 2:28 PM, Matt Davis wrote: > Hi all, > > Thank you for putting together this great tool! (I'll admit to being pretty blas? on IPython until you came out with the Notebook.) We've also got a PyCon proposal in on the topic of teaching with Notebooks. Very nice! > > There is already a lot of communication between SWC and the IPython devs so none of these requests are exactly news, but here are some things we'd like for the teaching side: > > - Integration with pythontutor.com Haha, I actually started to work on this yesterday. Not working yet, but it should be possible. The main challenge will be to allow the code run using opt to run in the same namespace as ipython. That is going to be messy and will require working closely with the opt authors. > - A way for an instructor to make a notebook then strip out code for distribution to students (so they can see our annotations but have to type their own code). So basically just remove all code cells, right? That won't be too difficult. > - A way to put hints into Markdown cells, e.g. collapsed by default but expandable if students want help. What about having md cells be collapsible and just show the first line when collapsed? > > I do pretty much all my in-person teaching with live typing but the slide show abilities also look quite cool and I'm sure we'll put those to use too! > > Thanks again for all your hard work! Hopefully one of these days I'll have some time to give back. > > Best, > > Matt Davis > > On Thu, Oct 4, 2012 at 1:16 PM, Brian Granger wrote: > HI, > > Software Carpentry just announced that they are going to transition > all of their materials to the IPython Notebook. This is a huge > victory for us, congrats everyone! > > Matthias - an important part of the story for them was having > notebooks viewable online using nbviewer - again, great work on that. > > Cheers, > > Brian > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Oct 4 17:57:48 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 4 Oct 2012 14:57:48 -0700 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: Hey, On Thu, Oct 4, 2012 at 2:28 PM, Matt Davis wrote: > Thank you for putting together this great tool! (I'll admit to being pretty > blas? on IPython until you came out with the Notebook.) We've also got a It's important to clarify why it took us so long to come out with the notebook: because we always viewed it as something that we wanted to be part of a fully coherent architecture. And we first had to build that foundation, which took us a *long* time. Once the foundation was in place, and with the years of false starts (failures we learned a lot from), then the notebook could develop naturally and fluidly. Brian started writing the first notebook code in early summer 2011 and by August I was demoing it at EuroScipy: that speed of development would have been impossible if the underlying foundation, architecture and protocols hadn't been something solid enough to put a heavy load on. Hopefully now the payoff of that approach is clear: a unified experience across clients, the ability to have things like the qt console or terminal supporting the same syntax, a protocol that enables third-party uses (the Julia guys just announced publicly they'll use ipython notebook for Julia - https://github.com/JuliaLang/julia/issues/1330), etc. > PyCon proposal in on the topic of teaching with Notebooks. Fantastic. > There is already a lot of communication between SWC and the IPython devs so > none of these requests are exactly news, but here are some things we'd like > for the teaching side: > > - Integration with pythontutor.com Min and I are meeting with Philip Guo, its author, in two weeks. We'll have a good chance to discuss all of this in person. > - A way for an instructor to make a notebook then strip out code for > distribution to students (so they can see our annotations but have to type > their own code). Very easy addition to nbconvert, and something we've had in mind from the start (actually with even more complex cases possible, such as generating both 'homework' and 'solution' versions of a notebook from a single source file). > - A way to put hints into Markdown cells, e.g. collapsed by default but > expandable if students want help. That's already possible, Matthias had an example lying around that he wrote for Josh Bloom at Scipy, or maybe it was Stefan... Ping... > I do pretty much all my in-person teaching with live typing but the slide > show abilities also look quite cool and I'm sure we'll put those to use too! For many contexts, the slideshow support will be very useful indeed. Cheers, f From benjaminrk at gmail.com Thu Oct 4 18:34:14 2012 From: benjaminrk at gmail.com (MinRK) Date: Thu, 4 Oct 2012 15:34:14 -0700 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: On Thu, Oct 4, 2012 at 2:41 PM, Brian Granger wrote: > Hi, > > Sent from my iPad > > On Oct 4, 2012, at 2:28 PM, Matt Davis wrote: > > Hi all, > > Thank you for putting together this great tool! (I'll admit to being > pretty blas? on IPython until you came out with the Notebook.) We've also > got a PyCon proposal in on the topic of teaching with Notebooks. > > > Very nice! > > > There is already a lot of communication between SWC and the IPython devs > so none of these requests are exactly news, but here are some things we'd > like for the teaching side: > > - Integration with pythontutor.com > > > Haha, I actually started to work on this yesterday. Not working yet, but > it should be possible. The main challenge will be to allow the code run > using opt to run in the same namespace as ipython. That is going to be > messy and will require working closely with the opt authors. > > - A way for an instructor to make a notebook then strip out code for > distribution to students (so they can see our annotations but have to type > their own code). > > > So basically just remove all code cells, right? That won't be too > difficult. > > - A way to put hints into Markdown cells, e.g. collapsed by default but > expandable if students want help. > > > What about having md cells be collapsible and just show the first line > when collapsed? > Here's a demo Notebook and script, which allow you to show/hide hint cells with buttons, as well as easily scrub solution code from your notebook: https://gist.github.com/3836889 > > > I do pretty much all my in-person teaching with live typing but the slide > show abilities also look quite cool and I'm sure we'll put those to use too! > > Thanks again for all your hard work! Hopefully one of these days I'll have > some time to give back. > > Best, > > Matt Davis > > On Thu, Oct 4, 2012 at 1:16 PM, Brian Granger wrote: > >> HI, >> >> Software Carpentry just announced that they are going to transition >> all of their materials to the IPython Notebook. This is a huge >> victory for us, congrats everyone! >> >> Matthias - an important part of the story for them was having >> notebooks viewable online using nbviewer - again, great work on that. >> >> Cheers, >> >> Brian >> >> -- >> Brian E. Granger >> Cal Poly State University, San Luis Obispo >> bgranger at calpoly.edu and ellisonbg at gmail.com >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Thu Oct 4 22:34:12 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 4 Oct 2012 19:34:12 -0700 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: Thanks for posting this Min. The moral of the story is this - writing Python code to manipulate notebooks is super simple! Cheers, Brian On Thu, Oct 4, 2012 at 3:34 PM, MinRK wrote: > > > On Thu, Oct 4, 2012 at 2:41 PM, Brian Granger wrote: >> >> Hi, >> >> Sent from my iPad >> >> On Oct 4, 2012, at 2:28 PM, Matt Davis wrote: >> >> Hi all, >> >> Thank you for putting together this great tool! (I'll admit to being >> pretty blas? on IPython until you came out with the Notebook.) We've also >> got a PyCon proposal in on the topic of teaching with Notebooks. >> >> >> Very nice! >> >> >> There is already a lot of communication between SWC and the IPython devs >> so none of these requests are exactly news, but here are some things we'd >> like for the teaching side: >> >> - Integration with pythontutor.com >> >> >> Haha, I actually started to work on this yesterday. Not working yet, but >> it should be possible. The main challenge will be to allow the code run >> using opt to run in the same namespace as ipython. That is going to be >> messy and will require working closely with the opt authors. >> >> - A way for an instructor to make a notebook then strip out code for >> distribution to students (so they can see our annotations but have to type >> their own code). >> >> >> So basically just remove all code cells, right? That won't be too >> difficult. >> >> - A way to put hints into Markdown cells, e.g. collapsed by default but >> expandable if students want help. >> >> >> What about having md cells be collapsible and just show the first line >> when collapsed? > > > Here's a demo Notebook and script, which allow you to show/hide hint cells > with buttons, as well as easily scrub solution code from your notebook: > > https://gist.github.com/3836889 > > >> >> >> >> I do pretty much all my in-person teaching with live typing but the slide >> show abilities also look quite cool and I'm sure we'll put those to use too! >> >> Thanks again for all your hard work! Hopefully one of these days I'll have >> some time to give back. >> >> Best, >> >> Matt Davis >> >> On Thu, Oct 4, 2012 at 1:16 PM, Brian Granger wrote: >>> >>> HI, >>> >>> Software Carpentry just announced that they are going to transition >>> all of their materials to the IPython Notebook. This is a huge >>> victory for us, congrats everyone! >>> >>> Matthias - an important part of the story for them was having >>> notebooks viewable online using nbviewer - again, great work on that. >>> >>> Cheers, >>> >>> Brian >>> >>> -- >>> Brian E. Granger >>> Cal Poly State University, San Luis Obispo >>> bgranger at calpoly.edu and ellisonbg at gmail.com >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From bussonniermatthias at gmail.com Fri Oct 5 03:41:18 2012 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Fri, 5 Oct 2012 09:41:18 +0200 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: Le 4 oct. 2012 ? 23:57, Fernando Perez a ?crit : > Hey, > >> - A way to put hints into Markdown cells, e.g. collapsed by default but >> expandable if students want help. > > That's already possible, Matthias had an example lying around that he > wrote for Josh Bloom at Scipy, or maybe it was Stefan... Ping? I though collapsible was Min's work with the 'hint' button. I did encrypted cells that could be decrypted with passwords. https://github.com/Carreau/ipython/tree/cryptcell Josh Bloom asked that to be able to ship notebook with solution and only publish the password later. -- Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From carl.input at gmail.com Fri Oct 5 04:27:27 2012 From: carl.input at gmail.com (Carl Smith) Date: Fri, 5 Oct 2012 09:27:27 +0100 Subject: [IPython-dev] Fwd: [Python Central] Python book In-Reply-To: <506e9687d715a_59a2113c29c8725a@jobs-08.mail> References: <506e9687d715a_59a2113c29c8725a@jobs-08.mail> Message-ID: Hi all I just received this email from Jackson at Python Central, who I do a bit of blogging for. It's a mail he forwarded to the Python Central authors, apparently from a publishing company. I just thought I'd pass it on here in case anyone was interested in what they're doing. The email seems genuine, in that the domain is registered to Packt Publishing, and they seem to have works on Safari Books. Maybe a letter from Fernando could convince them to base their *Python Data Visualization Cookbook* on IPython Notebook. I dunno, it's early here ~ I need another cup of tea before I start thinking about anything ~ just a heads-up really. Cheers Carl ---------- Forwarded message ---------- From: Jackson Cooper Date: 5 October 2012 09:12 Subject: [Python Central] Python book To: Carl Smith Reply ABOVE THIS LINE to add a comment to this message Project: Python Central Company: Aztech Media Pty Ltd Jackson Cooper posted a new message: Python book Hi all I got an email from someone called Shrutika Kalbag from a book publishing company called Packt Publishing, as follows: My name is Shrutika Kalbag and I'm an Author Relationship Executive at Packt Publishing. Packt is an IT publishing company that publishes focused IT and allied technology books. *We are planning to publish an book on Python, titled, Python Data Visualization Cookbook and I am looking for potential authors to write this book. * *This book is about visualizing data with a selection of popular Python tools.* *I came across your website on Python and see you are experienced in this domain. Your expertise will be valuable to our readers.* *I look forward to the possibility of working together to publish this book and make it a success.* *Do let me know if you are interested in this project and if you have any queries or doubts.* * * If you're interested, contact her here: shrutikak at packtpub.com. This message was sent to Carl Smith, Jackson Cooper, Joanna Alonzo, Les D, Makkalot Makkalot, Nathan Smith, Sergio Tapia, Shafiul Azam, and Taka Muraoka. Stop receiving emailswhen comments are posted to this message. Prefer plain text emails? Delivered by Basecamp -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.m.bray at gmail.com Fri Oct 5 14:38:59 2012 From: erik.m.bray at gmail.com (Erik Bray) Date: Fri, 5 Oct 2012 14:38:59 -0400 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: On Thu, Oct 4, 2012 at 5:41 PM, Brian Granger wrote: > - A way for an instructor to make a notebook then strip out code for > distribution to students (so they can see our annotations but have to type > their own code). > > > So basically just remove all code cells, right? That won't be too > difficult. Maybe. More useful, for me anyways, would be to leave the code cells in but strip everything out of them except for comments. I've been thinking of writing a little script to do this, because I have a couple notebooks I would like to do that to for the purposes of teaching. For simplicity's sake it would only work on code cells that actually contain Python code :) Erik From benjaminrk at gmail.com Fri Oct 5 15:38:58 2012 From: benjaminrk at gmail.com (MinRK) Date: Fri, 5 Oct 2012 12:38:58 -0700 Subject: [IPython-dev] SWC using IPython Notebook In-Reply-To: References: Message-ID: On Fri, Oct 5, 2012 at 11:38 AM, Erik Bray wrote: > On Thu, Oct 4, 2012 at 5:41 PM, Brian Granger wrote: > > - A way for an instructor to make a notebook then strip out code for > > distribution to students (so they can see our annotations but have to > type > > their own code). > > > > > > So basically just remove all code cells, right? That won't be too > > difficult. > > Maybe. More useful, for me anyways, would be to leave the code cells > in but strip everything out of them except for comments. I've been > thinking of writing a little script to do this, because I have a > couple notebooks I would like to do that to for the purposes of > teaching. For simplicity's sake it would only work on code cells that > actually contain Python code :) > The script I linked above is close: https://gist.github.com/3836889 It empties out code cells that start with a particular comment. It should be straightforward to modify it to perform any transformation you want. -MinRK > > Erik > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewturk at gmail.com Fri Oct 5 17:31:57 2012 From: matthewturk at gmail.com (Matthew Turk) Date: Fri, 5 Oct 2012 14:31:57 -0700 Subject: [IPython-dev] Push cell contents to the notebook Message-ID: Hi all, Is there a way to push cell contents to a notebook? (Without executing them.) For instance, this would be useful for grabbing a script from a pastebin and appending it as an additional cell. Thanks! -Matt From takowl at gmail.com Fri Oct 5 18:08:17 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 5 Oct 2012 23:08:17 +0100 Subject: [IPython-dev] Push cell contents to the notebook In-Reply-To: References: Message-ID: On 5 October 2012 22:31, Matthew Turk wrote: > Is there a way to push cell contents to a notebook? (Without > executing them.) For instance, this would be useful for grabbing a > script from a pastebin and appending it as an additional cell. I think what you're after is the %load magic command. You can do this: %load http://example.com/12345/foo.py It also works with local files. Thomas From matthewturk at gmail.com Fri Oct 5 18:13:30 2012 From: matthewturk at gmail.com (Matthew Turk) Date: Fri, 5 Oct 2012 15:13:30 -0700 Subject: [IPython-dev] Push cell contents to the notebook In-Reply-To: References: Message-ID: Hi Thomas, On Fri, Oct 5, 2012 at 3:08 PM, Thomas Kluyver wrote: > On 5 October 2012 22:31, Matthew Turk wrote: >> Is there a way to push cell contents to a notebook? (Without >> executing them.) For instance, this would be useful for grabbing a >> script from a pastebin and appending it as an additional cell. > > I think what you're after is the %load magic command. You can do this: > > %load http://example.com/12345/foo.py > > It also works with local files. That's exactly what I was looking for. I knew about loading local files, but I didn't realize it could load remote files! Thank you! Also, I realized now I sent this to the developers' list instead of the users' list -- apologies for that. -Matt > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From j.reid at mail.cryst.bbk.ac.uk Sat Oct 6 12:24:44 2012 From: j.reid at mail.cryst.bbk.ac.uk (John Reid) Date: Sat, 06 Oct 2012 17:24:44 +0100 Subject: [IPython-dev] Ipython unicode error Message-ID: Apologies if this is a duplicate post, the first one didn't seem to appear... Hi, I tried to do a ls from within ipython and it exited: In [30]: ls Output/scan-T=001 Traceback (most recent call last): File "/usr/bin/ipython", line 8, in launch_new_instance() File "/usr/lib/python2.7/dist-packages/IPython/frontend/terminal/ipapp.py", line 403, in launch_new_instance app.start() File "/usr/lib/python2.7/dist-packages/IPython/frontend/terminal/ipapp.py", line 377, in start self.shell.mainloop() File "/usr/lib/python2.7/dist-packages/IPython/frontend/terminal/interactiveshell.py", line 290, in mainloop self.interact(display_banner=display_banner) File "/usr/lib/python2.7/dist-packages/IPython/frontend/terminal/interactiveshell.py", line 409, in interact self.run_cell(source_raw, store_history=True) File "/usr/lib/python2.7/dist-packages/IPython/core/interactiveshell.py", line 2393, in run_cell self.logger.log(cell, raw_cell) File "/usr/lib/python2.7/dist-packages/IPython/core/logger.py", line 183, in log self.log_write(line_mod) File "/usr/lib/python2.7/dist-packages/IPython/core/logger.py", line 195, in log_write write(data) UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 0: ordinal not in range(128) If you suspect this is an IPython bug, please report it at: https://github.com/ipython/ipython/issues or send an email to the mailing list at ipython-dev at scipy.org You can print a more detailed traceback right now with "%tb", or use "%debug" to interactively debug it. Extra-detailed tracebacks for bug-reporting purposes can be enabled via: %config Application.verbose_crash=True I'm running 0.12.1. Is this a known problem? I'm reporting it as requested. Let me know if you want more info. John. From takowl at gmail.com Sat Oct 6 14:19:29 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Sat, 6 Oct 2012 19:19:29 +0100 Subject: [IPython-dev] Ipython unicode error In-Reply-To: References: Message-ID: Hi John, On 6 October 2012 17:24, John Reid wrote: > File "/usr/lib/python2.7/dist-packages/IPython/core/logger.py", line > 195, in log_write > write(data) > UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in > position 0: ordinal not in range(128) Thanks for the report, I think that looks similar to this error: https://github.com/ipython/ipython/issues/1777 The fix for that is in 0.13 - can you try upgrading? If you're using Ubuntu, you can use this PPA: https://launchpad.net/~jtaylor/+archive/ipython Thanks, Thomas From fperez.net at gmail.com Mon Oct 8 18:35:38 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 8 Oct 2012 15:35:38 -0700 Subject: [IPython-dev] IPython Hydra from Stanford Message-ID: Hey folks, our friends at the Stanford Center for Cognitive and Neurobiological Imaging (CNI), the team behind http://ipython.stanford.edu, just put up the script collection they wrote to deploy their notebook server: https://github.com/cni/ipython-hydra I had suggested the name ipython-ghettoblaster, but Bob is too much of a gentleman to go along with that :) This is a collection of scripts to let you deploy a notebook server in a group unix environment, hiding the drudgery of port management from your users. Since we know for a fact that a real multiuser solution for IPython will take some time to develop, this is an excellent stop-gap solution. We hope a cleaner solution will happen by next year, but Bob needed something right away, so a few weeks ago we had a little brainstorm about this and his team quickly came back with a working implementation. We hope this will serve people who'd like to deploy the notebook for their research group or a small set of students, for example, in an environment where they can assume that everyone already has Unix accounts. Many thanks to Bob and the rest of the CNI team at Stanford for making this available, I suspect it will be quite useful until we can build a production solution in IPython itself. Cheers, f From fperez.net at gmail.com Mon Oct 8 19:13:53 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 8 Oct 2012 16:13:53 -0700 Subject: [IPython-dev] IPython Hydra from Stanford In-Reply-To: References: Message-ID: On Mon, Oct 8, 2012 at 3:35 PM, Fernando Perez wrote: > our friends at the Stanford Center for Cognitive and Neurobiological > Imaging (CNI), the team behind http://ipython.stanford.edu, ps - for the curious, I mistyped the URL as it's https://, but furthermore, that only works if you're inside the Stanford network or VPN. So you can't really see it from the outside, sorry :) From nawijn at gmail.com Thu Oct 11 10:03:25 2012 From: nawijn at gmail.com (Marco Nawijn) Date: Thu, 11 Oct 2012 16:03:25 +0200 Subject: [IPython-dev] Proposal for small change in dir2() function to handle XMLRPC Server Proxies Message-ID: Dear All, I would to propose that a small change is made to the dir2() function in IPython to better handle tab completion on a xmlrpclib.ServerProxy object. Currently if you do tab completion on a xmlrpclib.ServerProxy it a traceback like the following is generated: === BEGIN TRACEBACK ===== Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/IPython/core/completer.py", line 847, in complete self.matches.extend(matcher(text)) File "/usr/lib/python2.7/site-packages/IPython/core/completer.py", line 641, in python_matches matches = self.attr_matches(text) File "/usr/lib/python2.7/site-packages/IPython/core/completer.py", line 393, in attr_matches words = dir2(obj) File "/usr/lib/python2.7/site-packages/IPython/utils/dir2.py", line 64, in dir2 words.extend(obj.trait_names()) File "/usr/lib64/python2.7/xmlrpclib.py", line 1224, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1578, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1264, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1297, in single_request return self.parse_response(response) File "/usr/lib64/python2.7/xmlrpclib.py", line 1473, in parse_response return u.close() File "/usr/lib64/python2.7/xmlrpclib.py", line 793, in close raise Fault(**self._stack[0]) Fault: :method "trait_names" is not supported'> If you suspect this is an IPython bug, please report it at: https://github.com/ipython/ipython/issues or send an email to the mailing list at ipython-dev at scipy.org You can print a more detailed traceback right now with "%tb", or use "%debug" to interactively debug it. Extra-detailed tracebacks for bug-reporting purposes can be enabled via: %config Application.verbose_crash=True ===== END TRACEBACK ====== The source of this as can be seen from the traceback is in the dir2() function. Since the main purpose of dir2 seems to be to handle border cases, I would like to propose that an additional case is implemented to handle xmlrpclib.ServerProxy objects. I have implemented it (as a proof of concept) as follows: import xmlrpclib if isinstance(obj, xmlrpclib.ServerProxy): words = obj.system.listMethods() return words else: words = dir(obj) <- this is already there It works like a charm :). Any comments on this? Kind regards, Marco Nawijn From jason-sage at creativetrax.com Thu Oct 11 15:56:45 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 11 Oct 2012 14:56:45 -0500 Subject: [IPython-dev] IPython Hydra from Stanford In-Reply-To: References: Message-ID: <5077247D.4020508@creativetrax.com> On 10/8/12 5:35 PM, Fernando Perez wrote: > We hope a cleaner solution will happen by next year Are there definitive plans to work on the multi-user aspect (e.g., committed resources or a rough schedule)? I've been thinking a lot lately about the future of the Sage notebook as well, and one idea is to pursue a replacement based on the ipython notebook. Thanks, Jason From jtaylor.debian at googlemail.com Thu Oct 11 16:04:13 2012 From: jtaylor.debian at googlemail.com (Julian Taylor) Date: Thu, 11 Oct 2012 22:04:13 +0200 Subject: [IPython-dev] 0.13.1 In-Reply-To: References: <72959517721A4CACA7384B462A944B70@gmail.com> Message-ID: <5077263D.4060504@googlemail.com> On 08/31/2012 05:39 AM, Fernando Perez wrote: > On Thu, Aug 30, 2012 at 3:09 PM, Bradley M. Froehle > wrote: >> I'm in favor of deferring the event loop issue to 0.14. > > I've just merged #2365, which was an easy fix. But I agree with you on > this one: event loop code is super delicate, so any fixes on that > front should be made early in the release cycle to have ample time for > field testing. Since a lot of problems with that type of code only > manifest themselves under interactive usage, we really shouldn't be > merging any such changes close to a release. > > Thanks a ton to Min for keeping 0.13.1 moving forward! We're not > likely to see 0.14 for a couple more months, so having a solid version > of 0.13 out will be great. > Any news on the 0.13.1 release? I just got reply from the debian release team that its suitable for wheezy but it should be done soonish. Unfortunately it might be a bit late for ubuntu 12.10 which closes on tuesday. From benjaminrk at gmail.com Thu Oct 11 16:21:48 2012 From: benjaminrk at gmail.com (MinRK) Date: Thu, 11 Oct 2012 13:21:48 -0700 Subject: [IPython-dev] Last call for 0.13.1 Message-ID: There is currently only one (trivial) PR outstanding for 0.13.1, so I think this is last call for anything that should go in before we cut a release. If anyone thinks that something not already covered belongs in an 0.13.1 release, speak up now. Obviously, if anything doesn't make it, there's no reason we can't have 0.13.2 in another month or two. -MinRK -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdroe at stsci.edu Thu Oct 11 17:49:14 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Thu, 11 Oct 2012 17:49:14 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser Message-ID: <50773EDA.4070609@stsci.edu> I have a proof-of-concept way to make interactive plots in the browser work using transparent PNGs described here: http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ No PRs yet, because this is miles from ready for that, but it would be helpful to get some feedback about how this works in different browsers/platforms/network environments etc. Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason-sage at creativetrax.com Thu Oct 11 17:57:00 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 11 Oct 2012 16:57:00 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50773EDA.4070609@stsci.edu> References: <50773EDA.4070609@stsci.edu> Message-ID: <507740AC.4070409@creativetrax.com> On 10/11/12 4:49 PM, Michael Droettboom wrote: > I have a proof-of-concept way to make interactive plots in the browser > work using transparent PNGs described here: > > http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ > > No PRs yet, because this is miles from ready for that, but it would be > helpful to get some feedback about how this works in different > browsers/platforms/network environments etc. Would you mind CCing the sage-notebook mailing list [1] on these sorts of posts? We're *very* interested in having this capability as well (especially in augmenting the Sage interact functionality). I'll forward this message, but bringing the sage-notebook community into the discussion would be more natural than me pushing messages back and forth between the lists. Thanks, Jason [1] https://groups.google.com/forum/?fromgroups#!forum/sage-notebook From jason-sage at creativetrax.com Thu Oct 11 18:08:19 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 11 Oct 2012 17:08:19 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50773EDA.4070609@stsci.edu> References: <50773EDA.4070609@stsci.edu> Message-ID: <50774353.1090800@creativetrax.com> On 10/11/12 4:49 PM, Michael Droettboom wrote: > I have a proof-of-concept way to make interactive plots in the browser > work using transparent PNGs described here: > > http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ > > No PRs yet, because this is miles from ready for that, but it would be > helpful to get some feedback about how this works in different > browsers/platforms/network environments etc. When I try to run this with matplotlib 1.1.0, I get: Traceback (most recent call last): File "/Users/grout/sage-trees/sage-5.4.rc1/local/lib/python2.7/site-packages/tornado-2.4-py2.7.egg/tornado/web.py", line 1042, in _execute getattr(self, self.request.method.lower())(*args, **kwargs) File "/Users/grout/projects/matplotlib-live/serve_figure.py", line 174, in get np.frombuffer(renderer.buffer_rgba(), dtype=np.uint32), TypeError: buffer_rgba() takes exactly 3 arguments (1 given) and the image doesn't show up. Is this something that is fixed in an matplotlib upgrade? Thanks, Jason From jason-sage at creativetrax.com Thu Oct 11 18:14:16 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 11 Oct 2012 17:14:16 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50774353.1090800@creativetrax.com> References: <50773EDA.4070609@stsci.edu> <50774353.1090800@creativetrax.com> Message-ID: <507744B8.5070706@creativetrax.com> On 10/11/12 5:08 PM, Jason Grout wrote: > On 10/11/12 4:49 PM, Michael Droettboom wrote: >> I have a proof-of-concept way to make interactive plots in the browser >> work using transparent PNGs described here: >> >> http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ >> >> No PRs yet, because this is miles from ready for that, but it would be >> helpful to get some feedback about how this works in different >> browsers/platforms/network environments etc. > > When I try to run this with matplotlib 1.1.0, I get: > > Traceback (most recent call last): > File > "/Users/grout/sage-trees/sage-5.4.rc1/local/lib/python2.7/site-packages/tornado-2.4-py2.7.egg/tornado/web.py", > line 1042, in _execute > getattr(self, self.request.method.lower())(*args, **kwargs) > File "/Users/grout/projects/matplotlib-live/serve_figure.py", line > 174, in get > np.frombuffer(renderer.buffer_rgba(), dtype=np.uint32), > TypeError: buffer_rgba() takes exactly 3 arguments (1 given) > > and the image doesn't show up. Is this something that is fixed in an > matplotlib upgrade? > Making the call buffer_rgba(0,0) seems to fix it: https://gist.github.com/3875846 Thanks, Jason From jason-sage at creativetrax.com Thu Oct 11 18:50:01 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 11 Oct 2012 17:50:01 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50773EDA.4070609@stsci.edu> References: <50773EDA.4070609@stsci.edu> Message-ID: <50774D19.1030900@creativetrax.com> On 10/11/12 4:49 PM, Michael Droettboom wrote: > I have a proof-of-concept way to make interactive plots in the browser > work using transparent PNGs described here: > > http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ > > No PRs yet, because this is miles from ready for that, but it would be > helpful to get some feedback about how this works in different > browsers/platforms/network environments etc. > A sample implementation using websockets instead of polling is here: https://gist.github.com/3875846 It still requests the file, which causes a delay. I think doing a png diff sounds like a great idea. What if we also transfer the png diff over the websocket connection (maybe in a binary frame)? Thanks, Jason From fperez.net at gmail.com Thu Oct 11 18:49:41 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 11 Oct 2012 15:49:41 -0700 Subject: [IPython-dev] IPython Hydra from Stanford In-Reply-To: <5077247D.4020508@creativetrax.com> References: <5077247D.4020508@creativetrax.com> Message-ID: Hi Jason, On Thu, Oct 11, 2012 at 12:56 PM, Jason Grout wrote: > Are there definitive plans to work on the multi-user aspect (e.g., > committed resources or a rough schedule)? I've been thinking a lot > lately about the future of the Sage notebook as well, and one idea is to > pursue a replacement based on the ipython notebook. absolutely. I've barely slept this week putting the final touches on a grant proposal that, if funded, will support *precisely* that, it's specific aim #1 of the proposal. If things go well, by Jan 1 we'll have resources to tackle this full-time, if not we'll do our best at the slower pace that we normally work. But it's definitely one of our highest priorities, it's just that the combination of me being offline due to John Hunter's illness/death, and working on three separate grants that involve IPython, has consumed all available bandwidth for a while. But as these grants get sent out, we'll begin clearing time for development work again. Cheers, f From benjaminrk at gmail.com Thu Oct 11 18:58:07 2012 From: benjaminrk at gmail.com (MinRK) Date: Thu, 11 Oct 2012 15:58:07 -0700 Subject: [IPython-dev] Last call for 0.13.1 In-Reply-To: References: Message-ID: On Thu, Oct 11, 2012 at 1:21 PM, MinRK wrote: > There is currently only one (trivial) PR outstanding for 0.13.1, so I > think this is last call for anything that should go in before we cut a > release. > > If anyone thinks that something not already covered belongs in an 0.13.1 > release, speak up now. > > Obviously, if anything doesn't make it, there's no reason we can't have > 0.13.2 in another month or two. > While we are considering last call for 0.13.1 and any further tweaks, I cut a first release candidate from the current state: http://archive.ipython.org/testing/0.13.1/ So we can get started with installation testing. This is just a bugfix release, so no new features to play with. -MinRK -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Oct 11 19:20:47 2012 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 11 Oct 2012 16:20:47 -0700 Subject: [IPython-dev] Last call for 0.13.1 In-Reply-To: References: Message-ID: Min, thanks a ton for your push on this! It would be great if we can get this in time for Ubuntu 12.10, whose cutoff date is Tuesday as indicated by Julian. I have some time this weekend to help out with a mini release party if you want... Cheers, f On Thu, Oct 11, 2012 at 3:58 PM, MinRK wrote: > > > On Thu, Oct 11, 2012 at 1:21 PM, MinRK wrote: >> >> There is currently only one (trivial) PR outstanding for 0.13.1, so I >> think this is last call for anything that should go in before we cut a >> release. >> >> If anyone thinks that something not already covered belongs in an 0.13.1 >> release, speak up now. >> >> Obviously, if anything doesn't make it, there's no reason we can't have >> 0.13.2 in another month or two. > > > While we are considering last call for 0.13.1 and any further tweaks, I cut > a first release candidate from the current state: > > http://archive.ipython.org/testing/0.13.1/ > > So we can get started with installation testing. > > This is just a bugfix release, so no new features to play with. > > -MinRK > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From asmeurer at gmail.com Thu Oct 11 19:22:27 2012 From: asmeurer at gmail.com (Aaron Meurer) Date: Thu, 11 Oct 2012 17:22:27 -0600 Subject: [IPython-dev] Last call for 0.13.1 In-Reply-To: References: Message-ID: You may want to start a deprecation cycle for the SymPy printing extension, which has been moved to SymPy itself as of 0.7.2. I would just have the following logic: if SymPy version is 0.7.2 or greater: use the ipython version (or should it use the sympy version?), but emit a deprecation warning telling the user to just load sympy.interactive.ipythonprinting if SymPy version is less than 0.7.2: don't do any warnings And then, when it's been long enough that probably most people are using SymPy 0.7.2 and IPython 0.13.1 or greater, remove it entirely. Aaron Meurer On Thu, Oct 11, 2012 at 4:58 PM, MinRK wrote: > > > On Thu, Oct 11, 2012 at 1:21 PM, MinRK wrote: >> >> There is currently only one (trivial) PR outstanding for 0.13.1, so I >> think this is last call for anything that should go in before we cut a >> release. >> >> If anyone thinks that something not already covered belongs in an 0.13.1 >> release, speak up now. >> >> Obviously, if anything doesn't make it, there's no reason we can't have >> 0.13.2 in another month or two. > > > While we are considering last call for 0.13.1 and any further tweaks, I cut > a first release candidate from the current state: > > http://archive.ipython.org/testing/0.13.1/ > > So we can get started with installation testing. > > This is just a bugfix release, so no new features to play with. > > -MinRK > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From mdroe at stsci.edu Thu Oct 11 19:57:40 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Thu, 11 Oct 2012 19:57:40 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <507744B8.5070706@creativetrax.com> References: <50773EDA.4070609@stsci.edu> <50774353.1090800@creativetrax.com> <507744B8.5070706@creativetrax.com> Message-ID: <50775CF4.8090206@stsci.edu> On 10/11/2012 06:14 PM, Jason Grout wrote: > On 10/11/12 5:08 PM, Jason Grout wrote: >> On 10/11/12 4:49 PM, Michael Droettboom wrote: >>> I have a proof-of-concept way to make interactive plots in the browser >>> work using transparent PNGs described here: >>> >>> http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ >>> >>> No PRs yet, because this is miles from ready for that, but it would be >>> helpful to get some feedback about how this works in different >>> browsers/platforms/network environments etc. >> When I try to run this with matplotlib 1.1.0, I get: >> >> Traceback (most recent call last): >> File >> "/Users/grout/sage-trees/sage-5.4.rc1/local/lib/python2.7/site-packages/tornado-2.4-py2.7.egg/tornado/web.py", >> line 1042, in _execute >> getattr(self, self.request.method.lower())(*args, **kwargs) >> File "/Users/grout/projects/matplotlib-live/serve_figure.py", line >> 174, in get >> np.frombuffer(renderer.buffer_rgba(), dtype=np.uint32), >> TypeError: buffer_rgba() takes exactly 3 arguments (1 given) >> >> and the image doesn't show up. Is this something that is fixed in an >> matplotlib upgrade? >> > Making the call buffer_rgba(0,0) seems to fix it: > > https://gist.github.com/3875846 > Ah, yes. That API was changed since 1.1. Your fix is fine for 1.1, but it will need to be changed for 1.2. Mike From jason-sage at creativetrax.com Thu Oct 11 22:06:23 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Thu, 11 Oct 2012 21:06:23 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50774D19.1030900@creativetrax.com> References: <50773EDA.4070609@stsci.edu> <50774D19.1030900@creativetrax.com> Message-ID: <50777B1F.5020501@creativetrax.com> On 10/11/12 5:50 PM, Jason Grout wrote: > On 10/11/12 4:49 PM, Michael Droettboom wrote: >> I have a proof-of-concept way to make interactive plots in the browser >> work using transparent PNGs described here: >> >> http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ >> >> No PRs yet, because this is miles from ready for that, but it would be >> helpful to get some feedback about how this works in different >> browsers/platforms/network environments etc. >> > > A sample implementation using websockets instead of polling is here: > > https://gist.github.com/3875846 > > It still requests the file, which causes a delay. I think doing a png > diff sounds like a great idea. What if we also transfer the png diff > over the websocket connection (maybe in a binary frame)? I just updated it to transfer the image over websocket too. More timing tests need to be done to see which approach is better, especially tests across a network. The current version of https://gist.github.com/3875846 also transfers over a base64 version of the data; it seems like it would be better to transfer it as a binary message. Also, the polling isn't very smart compared to your original example. Right now, I have a continuous 50ms refresh, but of course, it should be smarter about requesting a refresh only when we know there is a change. Thanks, Jason From ellisonbg at gmail.com Thu Oct 11 23:45:51 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 11 Oct 2012 20:45:51 -0700 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50773EDA.4070609@stsci.edu> References: <50773EDA.4070609@stsci.edu> Message-ID: It is not clear to me that the stream of PNGs will win in the end. If you make a single static plot of a large data set, that is way better than trying to send the data to the browser and rendering it there. But if you have to send hundreds or thousands of PNGs to get interactivity, that benefit may be washed out. Especially if you have multiple users interacting with plots - the server could quickly grind to a halt. I think we should do tests to see how bad it gets, taking into account the multiple user question. The one performance benefit that I can think of is that you can tune the level of interactivity to limit the data that comes back. For large data sets, users might be willing to settle for less interactivity. That option doesn't exist when you send all the data back. Cheers, Brian On Thu, Oct 11, 2012 at 2:49 PM, Michael Droettboom wrote: > I have a proof-of-concept way to make interactive plots in the browser work > using transparent PNGs described here: > > http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ > > No PRs yet, because this is miles from ready for that, but it would be > helpful to get some feedback about how this works in different > browsers/platforms/network environments etc. > > Mike > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From jason-sage at creativetrax.com Fri Oct 12 02:39:28 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 12 Oct 2012 01:39:28 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: References: <50773EDA.4070609@stsci.edu> Message-ID: <5077BB20.5090704@creativetrax.com> On 10/11/12 10:45 PM, Brian Granger wrote: > It is not clear to me that the stream of PNGs will win in the end. What do you see as alternatives? Reimplementing matplotlib in the browser? SVG-based rendering, where we send over a complete svg image at each step (or just the changed svd elements)? I'm curious what other ideas you have. If > you make a single static plot of a large data set, that is way better > than trying to send the data to the browser and rendering it there. > But if you have to send hundreds or thousands of PNGs to get > interactivity, that benefit may be washed out. Especially if you have > multiple users interacting with plots - the server could quickly grind > to a halt. I think we should do tests to see how bad it gets, taking > into account the multiple user question. The one performance benefit > that I can think of is that you can tune the level of interactivity to > limit the data that comes back. For large data sets, users might be > willing to settle for less interactivity. That option doesn't exist > when you send all the data back. I implemented server-side rate-limiting (see my most recent modifications at https://gist.github.com/3875846, for example). Or are you saying that the server might not be able to render pngs for hundreds or thousands of users? I wonder if we can get a better protocol than just a "Motion PNG" [1]. A disappointing thing is that it seems to take about 100ms to render the example png image; it would sure be nice if that was a lot faster. [1] http://en.wikipedia.org/wiki/Motion_JPEG From takowl at gmail.com Fri Oct 12 07:25:31 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 12 Oct 2012 12:25:31 +0100 Subject: [IPython-dev] Proposal for small change in dir2() function to handle XMLRPC Server Proxies In-Reply-To: References: Message-ID: On 11 October 2012 15:03, Marco Nawijn wrote: > I would to propose that a small change is made to the dir2() function > in IPython to better handle tab completion on a xmlrpclib.ServerProxy > object. I've also run into this before, and I agree that we should handle it better, but I don't think special casing xmlrpclib objects is the answer. Apart from anything else, we don't want to import xmlrpclib in every process just to check for it in tab completions. >From IPython's end, I think we should do a better job of catching exceptions raised in tab completion. But I think it's up to xmlrpclib to make __dir__ work properly. Note that this is already improved in Python 3 - the implementation of dir() now looks up __dir__ in a way that doesn't go via __getattr__, so it falls back to the standard dir() method. Thomas From mdroe at stsci.edu Fri Oct 12 10:28:31 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Fri, 12 Oct 2012 10:28:31 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50777B1F.5020501@creativetrax.com> References: <50773EDA.4070609@stsci.edu> <50774D19.1030900@creativetrax.com> <50777B1F.5020501@creativetrax.com> Message-ID: <5078290F.7050709@stsci.edu> On 10/11/2012 10:06 PM, Jason Grout wrote: > On 10/11/12 5:50 PM, Jason Grout wrote: >> On 10/11/12 4:49 PM, Michael Droettboom wrote: >>> I have a proof-of-concept way to make interactive plots in the browser >>> work using transparent PNGs described here: >>> >>> http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ >>> >>> No PRs yet, because this is miles from ready for that, but it would be >>> helpful to get some feedback about how this works in different >>> browsers/platforms/network environments etc. >>> >> A sample implementation using websockets instead of polling is here: >> >> https://gist.github.com/3875846 >> >> It still requests the file, which causes a delay. I think doing a png >> diff sounds like a great idea. What if we also transfer the png diff >> over the websocket connection (maybe in a binary frame)? > I just updated it to transfer the image over websocket too. More timing > tests need to be done to see which approach is better, especially tests > across a network. Thanks. That is very helpful > The current version of > https://gist.github.com/3875846 also transfers over a base64 version of > the data; it seems like it would be better to transfer it as a binary > message. I see -- just so I'm clear: that's because we send it over a websocket, and it's not obvious how else to put the binary data into the Canvas Image object on the browser end? I see that that's a bit suboptimal. I wonder if it makes sense to use a socket for the event communication, but a regular request to get the image data. Then then "push" would come from the event websocket as in my original example. > > Also, the polling isn't very smart compared to your original example. > Right now, I have a continuous 50ms refresh, but of course, it should be > smarter about requesting a refresh only when we know there is a change. I don't see where this is happening -- it looks like the images are pushed by the server when it knows there is a change. I'm probably missing something... Thanks again, Mike > > Thanks, > > Jason > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From mdroe at stsci.edu Fri Oct 12 10:31:04 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Fri, 12 Oct 2012 10:31:04 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <5077BB20.5090704@creativetrax.com> References: <50773EDA.4070609@stsci.edu> <5077BB20.5090704@creativetrax.com> Message-ID: <507829A8.2040906@stsci.edu> On 10/12/2012 02:39 AM, Jason Grout wrote: > >> you make a single static plot of a large data set, that is way better >> than trying to send the data to the browser and rendering it there. >> But if you have to send hundreds or thousands of PNGs to get >> interactivity, that benefit may be washed out. Especially if you have >> multiple users interacting with plots - the server could quickly grind >> to a halt. I think we should do tests to see how bad it gets, taking >> into account the multiple user question. The one performance benefit >> that I can think of is that you can tune the level of interactivity to >> limit the data that comes back. For large data sets, users might be >> willing to settle for less interactivity. That option doesn't exist >> when you send all the data back. > I implemented server-side rate-limiting (see my most recent > modifications at https://gist.github.com/3875846, for example). Or are > you saying that the server might not be able to render pngs for hundreds > or thousands of users? I wonder if we can get a better protocol than > just a "Motion PNG" [1]. > > A disappointing thing is that it seems to take about 100ms to render the > example png image; it would sure be nice if that was a lot faster. Yeah -- but solving that has nothing to do with the the browser. I sense that's already quite close to optimal for what it's doing. > > > [1] http://en.wikipedia.org/wiki/Motion_JPEG > I had ruled this out initially due to the lossy artifacts and the much larger encoding overhead. But perhaps it could be provided as an alternative... I think Motion PNG makes sense locally, where bandwidth is high and CPU time is shared between the server and the client, and Motion JPEG might make more sense remotely. Mike From mdroe at stsci.edu Fri Oct 12 13:04:49 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Fri, 12 Oct 2012 13:04:49 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <5078290F.7050709@stsci.edu> References: <50773EDA.4070609@stsci.edu> <50774D19.1030900@creativetrax.com> <50777B1F.5020501@creativetrax.com> <5078290F.7050709@stsci.edu> Message-ID: <50784DB1.7040208@stsci.edu> On 10/12/2012 10:28 AM, Michael Droettboom wrote: > On 10/11/2012 10:06 PM, Jason Grout wrote: >> On 10/11/12 5:50 PM, Jason Grout wrote: >>> On 10/11/12 4:49 PM, Michael Droettboom wrote: >>>> I have a proof-of-concept way to make interactive plots in the browser >>>> work using transparent PNGs described here: >>>> >>>> http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ >>>> >>>> >>>> No PRs yet, because this is miles from ready for that, but it would be >>>> helpful to get some feedback about how this works in different >>>> browsers/platforms/network environments etc. >>>> >>> A sample implementation using websockets instead of polling is here: >>> >>> https://gist.github.com/3875846 >>> >>> It still requests the file, which causes a delay. I think doing a png >>> diff sounds like a great idea. What if we also transfer the png diff >>> over the websocket connection (maybe in a binary frame)? >> I just updated it to transfer the image over websocket too. More timing >> tests need to be done to see which approach is better, especially tests >> across a network. > > Thanks. That is very helpful > >> The current version of >> https://gist.github.com/3875846 also transfers over a base64 version of >> the data; it seems like it would be better to transfer it as a binary >> message. > > I see -- just so I'm clear: that's because we send it over a > websocket, and it's not obvious how else to put the binary data into > the Canvas Image object on the browser end? I see that that's a bit > suboptimal. I wonder if it makes sense to use a socket for the event > communication, but a regular request to get the image data. Then then > "push" would come from the event websocket as in my original example. I found a better way -- recent versions of the websockets API do support binary data. The magic incantation to put load binary data into an image object is here: https://github.com/binaryjs/binaryjs/blob/master/examples/helloworld/index.html I've created a new github repo for this -- if we're going to collaborate on this before it develops into a real patch against matplotlib and/or ipython and/or sage that should be more convenient than using gists. https://github.com/mdboom/mpl_browser_experiments Cheers, Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdroe at stsci.edu Fri Oct 12 14:20:57 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Fri, 12 Oct 2012 14:20:57 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: References: <50773EDA.4070609@stsci.edu> Message-ID: <50785F89.6090105@stsci.edu> Yes -- let's test this in as many environments and scenarios as we can. Feel free to check out and kick the tires and let me know how it works. https://github.com/mdboom/mpl_browser_experiments Thanks to Jason's work to make this work in websockets and some further tweaks today, the multi-user scenario is much improved. The difference image is computed only once per server and then sent out to all of the clients. So, yes, bandwidth usage increases O(n) (where n is the number of clients), but CPU time on the server should be relatively constant. It's conceivable that duplicating the data to multiple clients could be handled by some sort of multicast proxy. Obviously, if we were just able to send transformation changes to each client, the bandwidth would be much smaller. But that requires moving the rendering into the browser, or at least something local that contains all of the data. Mike On 10/11/2012 11:45 PM, Brian Granger wrote: > It is not clear to me that the stream of PNGs will win in the end. If > you make a single static plot of a large data set, that is way better > than trying to send the data to the browser and rendering it there. > But if you have to send hundreds or thousands of PNGs to get > interactivity, that benefit may be washed out. Especially if you have > multiple users interacting with plots - the server could quickly grind > to a halt. I think we should do tests to see how bad it gets, taking > into account the multiple user question. The one performance benefit > that I can think of is that you can tune the level of interactivity to > limit the data that comes back. For large data sets, users might be > willing to settle for less interactivity. That option doesn't exist > when you send all the data back. > > Cheers, > > Brian > > > > On Thu, Oct 11, 2012 at 2:49 PM, Michael Droettboom wrote: >> I have a proof-of-concept way to make interactive plots in the browser work >> using transparent PNGs described here: >> >> http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ >> >> No PRs yet, because this is miles from ready for that, but it would be >> helpful to get some feedback about how this works in different >> browsers/platforms/network environments etc. >> >> Mike >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nawijn at gmail.com Fri Oct 12 14:47:17 2012 From: nawijn at gmail.com (Marco Nawijn) Date: Fri, 12 Oct 2012 20:47:17 +0200 Subject: [IPython-dev] Proposal for small change in dir2() function to handle XMLRPC Server Proxies In-Reply-To: References: Message-ID: On Fri, Oct 12, 2012 at 1:25 PM, Thomas Kluyver wrote: > On 11 October 2012 15:03, Marco Nawijn wrote: >> I would to propose that a small change is made to the dir2() function >> in IPython to better handle tab completion on a xmlrpclib.ServerProxy >> object. > > I've also run into this before, and I agree that we should handle it > better, but I don't think special casing xmlrpclib objects is the > answer. Apart from anything else, we don't want to import xmlrpclib in > every process just to check for it in tab completions. > > >From IPython's end, I think we should do a better job of catching > exceptions raised in tab completion. But I think it's up to xmlrpclib > to make __dir__ work properly. > > Note that this is already improved in Python 3 - the implementation of > dir() now looks up __dir__ in a way that doesn't go via __getattr__, > so it falls back to the standard dir() method. > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev Hi Thomas, I agree with you that it would be better to change the xmlrpc Proxy object to better respond to __dir__, instead of changing dir2(). I will take a look at it and see if I can come up with an elegant solution. It was just so cool to see the code completion of the server side functionality from a client side terminal just by making this small change :). Thanks for the response and kind regards, Marco From ellisonbg at gmail.com Fri Oct 12 13:06:16 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 12 Oct 2012 10:06:16 -0700 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <5077BB20.5090704@creativetrax.com> References: <50773EDA.4070609@stsci.edu> <5077BB20.5090704@creativetrax.com> Message-ID: On Thu, Oct 11, 2012 at 11:39 PM, Jason Grout wrote: > On 10/11/12 10:45 PM, Brian Granger wrote: >> It is not clear to me that the stream of PNGs will win in the end. > > What do you see as alternatives? Reimplementing matplotlib in the > browser? SVG-based rendering, where we send over a complete svg image > at each step (or just the changed svd elements)? I'm curious what other > ideas you have. There are two cases: * Small data sets and relatively simple plots types. For these, I do think it makes sense to send the raw JSON data to the browser and implement a Javascript based plotting library that consumes the JSON and has an extremely high level of interactivity. I don't think we need to do *all* possible plot types with the pixel perfection, embedded math, infinite configurability. When people are working interactively they want: i) plots that look spectacular - pixelized images don't cut it and ii) high levels of interactivity. I am not suggesting that we reimplement all of Matplotlib. Keep in mind that current situation with Matplotlib and the IPython Notebook is not very ideal. The pngs that are the default don't look that great and svg is not yet good enough to use as the default. We really do need in browser plotting that *looks* better. Another important aspect of the JSON data approach. We want to enable Notebooks to be viewed and used in contexts outside the actual notebook interface (nbviewer for example) where there is *no* python code that is run. Because we save the JSON data in the notebook, JSON based plotting can be done interactively in contexts like nbviewer. For sharing computational results with a wide audience that is critical. * Large data sets I don't really have any alternatives - this is simply a really hard problem. In some contexts downsampling is possible. In other contexts it is possible to stream subsets of JSON data at are required for the current view. It is my impression that the folks at Continuum are working on this problem. Maybe they have some insight into this problem. > > If >> you make a single static plot of a large data set, that is way better >> than trying to send the data to the browser and rendering it there. >> But if you have to send hundreds or thousands of PNGs to get >> interactivity, that benefit may be washed out. Especially if you have >> multiple users interacting with plots - the server could quickly grind >> to a halt. I think we should do tests to see how bad it gets, taking >> into account the multiple user question. The one performance benefit >> that I can think of is that you can tune the level of interactivity to >> limit the data that comes back. For large data sets, users might be >> willing to settle for less interactivity. That option doesn't exist >> when you send all the data back. > > I implemented server-side rate-limiting (see my most recent > modifications at https://gist.github.com/3875846, for example). Or are > you saying that the server might not be able to render pngs for hundreds > or thousands of users? I wonder if we can get a better protocol than > just a "Motion PNG" [1]. > > A disappointing thing is that it seems to take about 100ms to render the > example png image; it would sure be nice if that was a lot faster. > > > [1] http://en.wikipedia.org/wiki/Motion_JPEG > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From jason-sage at creativetrax.com Fri Oct 12 12:35:15 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 12 Oct 2012 11:35:15 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50773EDA.4070609@stsci.edu> References: <50773EDA.4070609@stsci.edu> Message-ID: <507846C3.9010903@creativetrax.com> On 10/11/12 4:49 PM, Michael Droettboom wrote: > I have a proof-of-concept way to make interactive plots in the browser > work using transparent PNGs described here: > > http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ > > No PRs yet, because this is miles from ready for that, but it would be > helpful to get some feedback about how this works in different > browsers/platforms/network environments etc. I just want to point out that this may be another use-case for custom messages in IPython. For example, a matplotlib figure, when it is drawn, could send the initial png over the websocket as a display_data message. Matplotlib would then register a callback with IPython to handle any custom messages coming back from the javascript (routed according to the parent header). When the javascript detects an event, it sends a message back to IPython, IPython calls the appropriate matplotlib callback, and matplotlib has the chance to send a new figure over. The javascript side likewise has a callback registered to handle matplotlib's return messages. These messages back and forth may make sense to have custom message types. Thanks, Jason From mdroe at stsci.edu Fri Oct 12 15:21:28 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Fri, 12 Oct 2012 15:21:28 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: References: <50773EDA.4070609@stsci.edu> <5077BB20.5090704@creativetrax.com> Message-ID: <50786DB8.3040008@stsci.edu> On 10/12/2012 01:06 PM, Brian Granger wrote: > On Thu, Oct 11, 2012 at 11:39 PM, Jason Grout > wrote: >> On 10/11/12 10:45 PM, Brian Granger wrote: >>> It is not clear to me that the stream of PNGs will win in the end. >> What do you see as alternatives? Reimplementing matplotlib in the >> browser? SVG-based rendering, where we send over a complete svg image >> at each step (or just the changed svd elements)? I'm curious what other >> ideas you have. > There are two cases: > > * Small data sets and relatively simple plots types. For these, I do > think it makes sense to send the raw JSON data to the browser and > implement a Javascript based plotting library that consumes the JSON > and has an extremely high level of interactivity. I don't think we > need to do *all* possible plot types with the pixel perfection, > embedded math, infinite configurability. When people are working > interactively they want: i) plots that look spectacular - pixelized > images don't cut it and ii) high levels of interactivity. > I am not suggesting that we reimplement all of Matplotlib. I agree about the reduced interactivity -- but there is an interactivity/functionality tradeoff, as you point out. I used to just think that it was a matter of implementing the missing functionality in Javascript, but I have since come to realize there are just things it can't do in a reasonable way (multiple projection types, multiple image interpolation modes) without another quantum leap in browser functionality. As for image quality, the approach outlined here uses the native pixel resolution, so it shouldn't be any more pixelized than anything else displayed on screen. I understand that Safari on Macs with a retina display does upsampling of images by default, but with the appropriate CSS we should be able to address that. Is it the lack of subpixel sampling that is bothersome? We've always eschewed that in matplotlib because it makes the PNGs non-portable (different displays have different RGB orderings), but we could flip it on in this case. > > Keep in mind that current situation with Matplotlib and the IPython > Notebook is not very ideal. The pngs that are the default don't look > that great and svg is not yet good enough to use as the default. We > really do need in browser plotting that *looks* better. What doesn't look great about the PNGs? A lot of effort has gone in to antialiasing and hinting the fonts in the best possible way and rounding lines to pixel centers etc to make it look as good as possible. If there's specific shortcomings, they can be addressed, since matplotlib controls the entire rendering stack. > > Another important aspect of the JSON data approach. We want to enable > Notebooks to be viewed and used in contexts outside the actual > notebook interface (nbviewer for example) where there is *no* python > code that is run. Because we save the JSON data in the notebook, JSON > based plotting can be done interactively in contexts like nbviewer. > For sharing computational results with a wide audience that is > critical. > > * Large data sets > > I don't really have any alternatives - this is simply a really hard > problem. In some contexts downsampling is possible. In other > contexts it is possible to stream subsets of JSON data at are required > for the current view. It is my impression that the folks at Continuum > are working on this problem. Maybe they have some insight into this > problem. I was actually speaking to Travis Oliphant at length about this last week, and the upshot of that conversation was this present approach. Mike > > >> If >>> you make a single static plot of a large data set, that is way better >>> than trying to send the data to the browser and rendering it there. >>> But if you have to send hundreds or thousands of PNGs to get >>> interactivity, that benefit may be washed out. Especially if you have >>> multiple users interacting with plots - the server could quickly grind >>> to a halt. I think we should do tests to see how bad it gets, taking >>> into account the multiple user question. The one performance benefit >>> that I can think of is that you can tune the level of interactivity to >>> limit the data that comes back. For large data sets, users might be >>> willing to settle for less interactivity. That option doesn't exist >>> when you send all the data back. >> I implemented server-side rate-limiting (see my most recent >> modifications at https://gist.github.com/3875846, for example). Or are >> you saying that the server might not be able to render pngs for hundreds >> or thousands of users? I wonder if we can get a better protocol than >> just a "Motion PNG" [1]. >> >> A disappointing thing is that it seems to take about 100ms to render the >> example png image; it would sure be nice if that was a lot faster. >> >> >> [1] http://en.wikipedia.org/wiki/Motion_JPEG >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > From jason-sage at creativetrax.com Fri Oct 12 15:38:31 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 12 Oct 2012 14:38:31 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <5078290F.7050709@stsci.edu> References: <50773EDA.4070609@stsci.edu> <50774D19.1030900@creativetrax.com> <50777B1F.5020501@creativetrax.com> <5078290F.7050709@stsci.edu> Message-ID: <507871B7.4000604@creativetrax.com> On 10/12/12 9:28 AM, Michael Droettboom wrote: >> Also, the polling isn't very smart compared to your original example. >> Right now, I have a continuous 50ms refresh, but of course, it should be >> smarter about requesting a refresh only when we know there is a change. > > I don't see where this is happening -- it looks like the images are > pushed by the server when it knows there is a change. I'm probably > missing something... I kept working on the code, so that is fixed now :). You can look in the revision history for when I sent the email to see what I was referring to, if it matters. Jason From jason-sage at creativetrax.com Fri Oct 12 15:40:52 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 12 Oct 2012 14:40:52 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50784DB1.7040208@stsci.edu> References: <50773EDA.4070609@stsci.edu> <50774D19.1030900@creativetrax.com> <50777B1F.5020501@creativetrax.com> <5078290F.7050709@stsci.edu> <50784DB1.7040208@stsci.edu> Message-ID: <50787244.4020209@creativetrax.com> On 10/12/12 12:04 PM, Michael Droettboom wrote: > I found a better way -- recent versions of the websockets API do support > binary data. The magic incantation to put load binary data into an > image object is here: > > https://github.com/binaryjs/binaryjs/blob/master/examples/helloworld/index.html My main obstruction was figuring out how to get the image data into the canvas (correctly interpreted, etc). It seems that using putImage overwrites all pixels on the canvas, regardless of the transparency settings, at least from what I read. > > I've created a new github repo for this -- if we're going to collaborate > on this before it develops into a real patch against matplotlib and/or > ipython and/or sage that should be more convenient than using gists. > > https://github.com/mdboom/mpl_browser_experiments Great. I may be somewhat offline for a few days, but mid-next week I'll be looking at this more for sure. Thanks, Jason From mdroe at stsci.edu Fri Oct 12 15:49:01 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Fri, 12 Oct 2012 15:49:01 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50787244.4020209@creativetrax.com> References: <50773EDA.4070609@stsci.edu> <50774D19.1030900@creativetrax.com> <50777B1F.5020501@creativetrax.com> <5078290F.7050709@stsci.edu> <50784DB1.7040208@stsci.edu> <50787244.4020209@creativetrax.com> Message-ID: <5078742D.3000108@stsci.edu> On 10/12/2012 03:40 PM, Jason Grout wrote: > On 10/12/12 12:04 PM, Michael Droettboom wrote: >> I found a better way -- recent versions of the websockets API do support >> binary data. The magic incantation to put load binary data into an >> image object is here: >> >> https://github.com/binaryjs/binaryjs/blob/master/examples/helloworld/index.html > > My main obstruction was figuring out how to get the image data into the > canvas (correctly interpreted, etc). It seems that using putImage > overwrites all pixels on the canvas, regardless of the transparency > settings, at least from what I read. True -- but rather than using putImage, I continued to use drawImage, and it works like a charm. The weird bit is the way to get a special url to an in-memory binary object: imageObj.src = (window.URL || window.webkitURL).createObjectURL( evt.data); (where evt.data is a Blob object). > >> I've created a new github repo for this -- if we're going to collaborate >> on this before it develops into a real patch against matplotlib and/or >> ipython and/or sage that should be more convenient than using gists. >> >> https://github.com/mdboom/mpl_browser_experiments > Great. I may be somewhat offline for a few days, but mid-next week I'll > be looking at this more for sure. Great. BTW -- in the latest incantation, I have animation working. Mike > > Thanks, > > Jason > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From jason-sage at creativetrax.com Fri Oct 12 16:06:21 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 12 Oct 2012 15:06:21 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <507829A8.2040906@stsci.edu> References: <50773EDA.4070609@stsci.edu> <5077BB20.5090704@creativetrax.com> <507829A8.2040906@stsci.edu> Message-ID: <5078783D.9050306@creativetrax.com> On 10/12/12 9:31 AM, Michael Droettboom wrote: > I think Motion PNG makes sense locally, where bandwidth is high and CPU > time is shared between the server and the client, and Motion JPEG might > make more sense remotely. > I wasn't implying that we use MJPEG, just merely pointing out the precedence for encoding frames (except we use png and do differences...). Jason From jason-sage at creativetrax.com Fri Oct 12 16:06:50 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 12 Oct 2012 15:06:50 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <5078742D.3000108@stsci.edu> References: <50773EDA.4070609@stsci.edu> <50774D19.1030900@creativetrax.com> <50777B1F.5020501@creativetrax.com> <5078290F.7050709@stsci.edu> <50784DB1.7040208@stsci.edu> <50787244.4020209@creativetrax.com> <5078742D.3000108@stsci.edu> Message-ID: <5078785A.4080309@creativetrax.com> On 10/12/12 2:49 PM, Michael Droettboom wrote: > BTW -- in the latest incantation, I have animation working. Cool! Do you have an example? Thanks, Jason From ellisonbg at gmail.com Fri Oct 12 16:52:05 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 12 Oct 2012 13:52:05 -0700 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <50786DB8.3040008@stsci.edu> References: <50773EDA.4070609@stsci.edu> <5077BB20.5090704@creativetrax.com> <50786DB8.3040008@stsci.edu> Message-ID: On Fri, Oct 12, 2012 at 12:21 PM, Michael Droettboom wrote: > On 10/12/2012 01:06 PM, Brian Granger wrote: >> On Thu, Oct 11, 2012 at 11:39 PM, Jason Grout >> wrote: >>> On 10/11/12 10:45 PM, Brian Granger wrote: >>>> It is not clear to me that the stream of PNGs will win in the end. >>> What do you see as alternatives? Reimplementing matplotlib in the >>> browser? SVG-based rendering, where we send over a complete svg image >>> at each step (or just the changed svd elements)? I'm curious what other >>> ideas you have. >> There are two cases: >> >> * Small data sets and relatively simple plots types. For these, I do >> think it makes sense to send the raw JSON data to the browser and >> implement a Javascript based plotting library that consumes the JSON >> and has an extremely high level of interactivity. I don't think we >> need to do *all* possible plot types with the pixel perfection, >> embedded math, infinite configurability. When people are working >> interactively they want: i) plots that look spectacular - pixelized >> images don't cut it and ii) high levels of interactivity. >> I am not suggesting that we reimplement all of Matplotlib. > > I agree about the reduced interactivity -- but there is an > interactivity/functionality tradeoff, as you point out. I used to just > think that it was a matter of implementing the missing functionality in > Javascript, but I have since come to realize there are just things it > can't do in a reasonable way (multiple projection types, multiple image > interpolation modes) without another quantum leap in browser functionality. Absolutely, and i don't think it makes sense to try and do those things in JS. > As for image quality, the approach outlined here uses the native pixel > resolution, so it shouldn't be any more pixelized than anything else > displayed on screen. I understand that Safari on Macs with a retina > display does upsampling of images by default, but with the appropriate > CSS we should be able to address that. Is it the lack of subpixel > sampling that is bothersome? We've always eschewed that in matplotlib > because it makes the PNGs non-portable (different displays have > different RGB orderings), but we could flip it on in this case. > >> >> Keep in mind that current situation with Matplotlib and the IPython >> Notebook is not very ideal. The pngs that are the default don't look >> that great and svg is not yet good enough to use as the default. We >> really do need in browser plotting that *looks* better. > > What doesn't look great about the PNGs? A lot of effort has gone in to > antialiasing and hinting the fonts in the best possible way and rounding > lines to pixel centers etc to make it look as good as possible. If > there's specific shortcomings, they can be addressed, since matplotlib > controls the entire rendering stack. They simply don't look at clean at vector based plots, especially now that users can resize the images dynamically. We could increase the resolution of the PNGs, but because the PNGs are saved in the notebook files, those files could get large. We have tried to choose a resolution/size that looks decent, but that isn't overly big. Remember our user create notebook with up to many dozens of plots. It is not a shortcoming of matplotlib, just of non-vector image formats in general. >> >> Another important aspect of the JSON data approach. We want to enable >> Notebooks to be viewed and used in contexts outside the actual >> notebook interface (nbviewer for example) where there is *no* python >> code that is run. Because we save the JSON data in the notebook, JSON >> based plotting can be done interactively in contexts like nbviewer. >> For sharing computational results with a wide audience that is >> critical. >> >> * Large data sets >> >> I don't really have any alternatives - this is simply a really hard >> problem. In some contexts downsampling is possible. In other >> contexts it is possible to stream subsets of JSON data at are required >> for the current view. It is my impression that the folks at Continuum >> are working on this problem. Maybe they have some insight into this >> problem. > > I was actually speaking to Travis Oliphant at length about this last > week, and the upshot of that conversation was this present approach. Nice, I will hopefully have a chance to look at the code soon. > Mike > >> >> >>> If >>>> you make a single static plot of a large data set, that is way better >>>> than trying to send the data to the browser and rendering it there. >>>> But if you have to send hundreds or thousands of PNGs to get >>>> interactivity, that benefit may be washed out. Especially if you have >>>> multiple users interacting with plots - the server could quickly grind >>>> to a halt. I think we should do tests to see how bad it gets, taking >>>> into account the multiple user question. The one performance benefit >>>> that I can think of is that you can tune the level of interactivity to >>>> limit the data that comes back. For large data sets, users might be >>>> willing to settle for less interactivity. That option doesn't exist >>>> when you send all the data back. >>> I implemented server-side rate-limiting (see my most recent >>> modifications at https://gist.github.com/3875846, for example). Or are >>> you saying that the server might not be able to render pngs for hundreds >>> or thousands of users? I wonder if we can get a better protocol than >>> just a "Motion PNG" [1]. >>> >>> A disappointing thing is that it seems to take about 100ms to render the >>> example png image; it would sure be nice if that was a lot faster. >>> >>> >>> [1] http://en.wikipedia.org/wiki/Motion_JPEG >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From ellisonbg at gmail.com Fri Oct 12 16:53:08 2012 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 12 Oct 2012 13:53:08 -0700 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <507846C3.9010903@creativetrax.com> References: <50773EDA.4070609@stsci.edu> <507846C3.9010903@creativetrax.com> Message-ID: On Fri, Oct 12, 2012 at 9:35 AM, Jason Grout wrote: > On 10/11/12 4:49 PM, Michael Droettboom wrote: >> I have a proof-of-concept way to make interactive plots in the browser >> work using transparent PNGs described here: >> >> http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ >> >> No PRs yet, because this is miles from ready for that, but it would be >> helpful to get some feedback about how this works in different >> browsers/platforms/network environments etc. > > I just want to point out that this may be another use-case for custom > messages in IPython. For example, a matplotlib figure, when it is > drawn, could send the initial png over the websocket as a display_data > message. Matplotlib would then register a callback with IPython to > handle any custom messages coming back from the javascript (routed > according to the parent header). When the javascript detects an event, > it sends a message back to IPython, IPython calls the appropriate > matplotlib callback, and matplotlib has the chance to send a new figure > over. The javascript side likewise has a callback registered to handle > matplotlib's return messages. These messages back and forth may make > sense to have custom message types. We can already do this type of round tripping without any custom messages. That is not to say that they wouldn't be useful though, they might be. But they are not required. Cheers, Brian > Thanks, > > Jason > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From mdroe at stsci.edu Fri Oct 12 17:52:47 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Fri, 12 Oct 2012 17:52:47 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <5078785A.4080309@creativetrax.com> References: <50773EDA.4070609@stsci.edu> <50774D19.1030900@creativetrax.com> <50777B1F.5020501@creativetrax.com> <5078290F.7050709@stsci.edu> <50784DB1.7040208@stsci.edu> <50787244.4020209@creativetrax.com> <5078742D.3000108@stsci.edu> <5078785A.4080309@creativetrax.com> Message-ID: <5078912F.7040209@stsci.edu> On 10/12/2012 04:06 PM, Jason Grout wrote: > On 10/12/12 2:49 PM, Michael Droettboom wrote: >> BTW -- in the latest incantation, I have animation working. > Cool! Do you have an example? > Yep. example3.py in the mdboom/mpl_browser_experiments repository. Mike From takowl at gmail.com Fri Oct 12 18:44:48 2012 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 12 Oct 2012 23:44:48 +0100 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: References: <50773EDA.4070609@stsci.edu> <5077BB20.5090704@creativetrax.com> <50786DB8.3040008@stsci.edu> Message-ID: On 12 October 2012 21:52, Brian Granger wrote: > They simply don't look at clean at vector based plots, especially now > that users can resize the images dynamically. We could increase the > resolution of the PNGs, but because the PNGs are saved in the notebook > files, those files could get large. We have tried to choose a > resolution/size that looks decent, but that isn't overly big. With this proposal, couldn't we hook the resize handle up to the matplotlib rendering? So when the user resizes the image, it triggers matplotlib to re-render it at the new size. Then the resolution/file size trade-off is up to the user. Thomas From jason-sage at creativetrax.com Fri Oct 12 19:16:32 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 12 Oct 2012 18:16:32 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: References: <50773EDA.4070609@stsci.edu> <507846C3.9010903@creativetrax.com> Message-ID: <5078A4D0.8080706@creativetrax.com> On 10/12/12 3:53 PM, Brian Granger wrote: > On Fri, Oct 12, 2012 at 9:35 AM, Jason Grout > wrote: >> On 10/11/12 4:49 PM, Michael Droettboom wrote: >>> I have a proof-of-concept way to make interactive plots in the browser >>> work using transparent PNGs described here: >>> >>> http://mdboom.github.com/blog/2012/10/11/matplotlib-in-the-browser-its-coming/ >>> >>> No PRs yet, because this is miles from ready for that, but it would be >>> helpful to get some feedback about how this works in different >>> browsers/platforms/network environments etc. >> >> I just want to point out that this may be another use-case for custom >> messages in IPython. For example, a matplotlib figure, when it is >> drawn, could send the initial png over the websocket as a display_data >> message. Matplotlib would then register a callback with IPython to >> handle any custom messages coming back from the javascript (routed >> according to the parent header). When the javascript detects an event, >> it sends a message back to IPython, IPython calls the appropriate >> matplotlib callback, and matplotlib has the chance to send a new figure >> over. The javascript side likewise has a callback registered to handle >> matplotlib's return messages. These messages back and forth may make >> sense to have custom message types. > > We can already do this type of round tripping without any custom > messages. That is not to say that they wouldn't be useful though, > they might be. But they are not required. Oh good. What would the message type be from the browser back to matplotlib? A display_data message back to matplotlib? And how would matplotlib be able to act on that message? Thanks, Jason From jason-sage at creativetrax.com Fri Oct 12 19:17:49 2012 From: jason-sage at creativetrax.com (Jason Grout) Date: Fri, 12 Oct 2012 18:17:49 -0500 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <5078912F.7040209@stsci.edu> References: <50773EDA.4070609@stsci.edu> <50774D19.1030900@creativetrax.com> <50777B1F.5020501@creativetrax.com> <5078290F.7050709@stsci.edu> <50784DB1.7040208@stsci.edu> <50787244.4020209@creativetrax.com> <5078742D.3000108@stsci.edu> <5078785A.4080309@creativetrax.com> <5078912F.7040209@stsci.edu> Message-ID: <5078A51D.1090006@creativetrax.com> On 10/12/12 4:52 PM, Michael Droettboom wrote: > On 10/12/2012 04:06 PM, Jason Grout wrote: >> On 10/12/12 2:49 PM, Michael Droettboom wrote: >>> BTW -- in the latest incantation, I have animation working. >> Cool! Do you have an example? >> > Yep. example3.py in the mdboom/mpl_browser_experiments repository. Did you push to github? I don't see it there. Thanks, Jason From mdroe at stsci.edu Sat Oct 13 18:42:47 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Sat, 13 Oct 2012 18:42:47 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: References: <50773EDA.4070609@stsci.edu> <5077BB20.5090704@creativetrax.com> <50786DB8.3040008@stsci.edu> Message-ID: <5079EE67.6020100@stsci.edu> On 10/12/2012 06:44 PM, Thomas Kluyver wrote: > On 12 October 2012 21:52, Brian Granger wrote: >> They simply don't look at clean at vector based plots, especially now >> that users can resize the images dynamically. We could increase the >> resolution of the PNGs, but because the PNGs are saved in the notebook >> files, those files could get large. We have tried to choose a >> resolution/size that looks decent, but that isn't overly big. > With this proposal, couldn't we hook the resize handle up to the > matplotlib rendering? So when the user resizes the image, it triggers > matplotlib to re-render it at the new size. Then the resolution/file > size trade-off is up to the user. > Absolutely -- no reason we couldn't do that. Mike From mdroe at stsci.edu Sat Oct 13 18:42:58 2012 From: mdroe at stsci.edu (Michael Droettboom) Date: Sat, 13 Oct 2012 18:42:58 -0400 Subject: [IPython-dev] Interactive Matplotlib in the browser In-Reply-To: <5078A51D.1090006@creativetrax.com> References: <50773EDA.4070609@stsci.edu> <50774D19.1030900@creativetrax.com> <50777B1F.5020501@creativetrax.com> <5078290F.7050709@stsci.edu> <50784DB1.7040208@stsci.edu> <50787244.4020209@creativetrax.com> <5078742D.3000108@stsci.edu> <5078785A.4080309@creativetrax.com> <5078912F.7040209@stsci.edu> <5078A51D.1090006@creativetrax.com> Message-ID: <5079EE72.1030907@stsci.edu> On 10/12/2012 07:17 PM, Jason Grout wrote: > On 10/12/12 4:52 PM, Michael Droettboom wrote: >> On 10/12/2012 04:06 PM, Jason Grout wrote: >>> On 10/12/12 2:49 PM, Michael Droettboom wrote: >>>> BTW -- in the latest incantation, I have animation working. >>> Cool! Do you have an example? >>> >> Yep. example3.py in the mdboom/mpl_browser_experiments repository. > Did you push to github? I don't see it there. > Oops. Try now. Mike From benjaminrk at gmail.com Sun Oct 14 17:05:20 2012 From: benjaminrk at gmail.com (MinRK) Date: Sun, 14 Oct 2012 14:05:20 -0700 Subject: [IPython-dev] Last call for 0.13.1 In-Reply-To: References: Message-ID: To keep things moving along (and for packaging reasons), I am setting the deadline at 24 hours from now (2PM PDT / 11PM GMT Monday October 15) for any *already merged PRs* that should be backported (only candidate at this point is 2480 , which has not yet been merged). After that, only fixes to docs or catastrophic installation bugs will be allowed. We can then cut 0.13.1 final on Friday (10/19). So *please* test the latest 0.13.1 RC: http://archive.ipython.org/testing/0.13.1 and report any serious issues. -MinRK On Thu, Oct 11, 2012 at 4:22 PM, Aaron Meurer wrote: > You may want to start a deprecation cycle for the SymPy printing > extension, which has been moved to SymPy itself as of 0.7.2. I would > just have the following logic: > > if SymPy version is 0.7.2 or greater: use the ipython version (or > should it use the sympy version?), but emit a deprecation warning > telling the user to just load sympy.interactive.ipythonprinting > > if SymPy version is less than 0.7.2: don't do any warnings > > And then, when it's been long enough that probably most people are > using SymPy 0.7.2 and IPython 0.13.1 or greater, remove it entirely. > > Aaron Meurer > > On Thu, Oct 11, 2012 at 4:58 PM, MinRK wrote: > > > > > > On Thu, Oct 11, 2012 at 1:21 PM, MinRK wrote: > >> > >> There is currently only one (trivial) PR outstanding for 0.13.1, so I > >> think this is last call for anything that should go in before we cut a > >> release. > >> > >> If anyone thinks that something not already covered belongs in an 0.13.1 > >> release, speak up now. > >> > >> Obviously, if anything doesn't make it, there's no reason we can't have > >> 0.13.2 in another month or two. > > > > > > While we are considering last call for 0.13.1 and any further tweaks, I > cut > > a first release candidate from the current state: > > > > http://archive.ipython.org/testing/0.13.1/ > > > > So we can get started with installation testing. > > > > This is just a bugfix release, so no new features to play with. > > > > -MinRK > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Oct 15 18:25:29 2012 From: benjaminrk at gmail.com (MinRK) Date: Mon, 15 Oct 2012 15:25:29 -0700 Subject: [IPython-dev] Last call for 0.13.1 In-Reply-To: References: Message-ID: Okay, door is officially closed on 0.13.1, and rc3 is here: http://archive.ipython.org/testing/0.13.1 Unless people report critical installation issues, rc3 will become 0.13.1 Final this weekend. No new bugfixes will be accepted. -MinRK On Sun, Oct 14, 2012 at 2:05 PM, MinRK wrote: > To keep things moving along (and for packaging reasons), I am setting the > deadline at 24 hours from now (2PM PDT / 11PM GMT Monday October 15) for > any *already merged PRs* that should be backported (only candidate at this > point is 2480 , which has > not yet been merged). After that, only fixes to docs or catastrophic > installation bugs will be allowed. > > We can then cut 0.13.1 final on Friday (10/19). > > So *please* test the latest 0.13.1 RC: > http://archive.ipython.org/testing/0.13.1 > and report any serious issues. > > -MinRK > > > On Thu, Oct 11, 2012 at 4:22 PM, Aaron Meurer wrote: > >> You may want to start a deprecation cycle for the SymPy printing >> extension, which has been moved to SymPy itself as of 0.7.2. I would >> just have the following logic: >> >> if SymPy version is 0.7.2 or greater: use the ipython version (or >> should it use the sympy version?), but emit a deprecation warning >> telling the user to just load sympy.interactive.ipythonprinting >> >> if SymPy version is less than 0.7.2: don't do any warnings >> >> And then, when it's been long enough that probably most people are >> using SymPy 0.7.2 and IPython 0.13.1 or greater, remove it entirely. >> >> Aaron Meurer >> >> On Thu, Oct 11, 2012 at 4:58 PM, MinRK wrote: >> > >> > >> > On Thu, Oct 11, 2012 at 1:21 PM, MinRK wrote: >> >> >> >> There is currently only one (trivial) PR outstanding for 0.13.1, so I >> >> think this is last call for anything that should go in before we cut a >> >> release. >> >> >> >> If anyone thinks that something not already covered belongs in an >> 0.13.1 >> >> release, speak up now. >> >> >> >> Obviously, if anything doesn't make it, there's no reason we can't have >> >> 0.13.2 in another month or two. >> > >> > >> > While we are considering last call for 0.13.1 and any further tweaks, I >> cut >> > a first release candidate from the current state: >> > >> > http://archive.ipython.org/testing/0.13.1/ >> > >> > So we can get started with installation testing. >> > >> > This is just a bugfix release, so no new features to play with. >> > >> > -MinRK >> > >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cekees at gmail.com Thu Oct 18 17:59:33 2012 From: cekees at gmail.com (Chris Kees) Date: Thu, 18 Oct 2012 16:59:33 -0500 Subject: [IPython-dev] interactive widgets with d3 and three.js Message-ID: Hi, I'm trying to build some interactive representations of geometric objects (domains of pde's) in the notebook. For now I'm using d3.js and three.js for the interactive representation and haven't written any callbacks to actually change the python representations so this equations is just about display for now. I'm starting with rectangles and rectangular cuboids. I can't seem to get the objects to display and interact properly without separating the html from the javascript like this: display(d.html) #add a canvas or div element d.javascript #get the canvas or div element and render to it with d3.js or three.js Shouldn't I be able to just stick the javascript in a