From rgbkrk at gmail.com Fri Aug 1 00:23:18 2014 From: rgbkrk at gmail.com (Kyle Kelley) Date: Thu, 31 Jul 2014 23:23:18 -0500 Subject: [IPython-dev] CVE Request: Enforce use of HTTPS for MathJax in IPython Message-ID: All, We would like to request a CVE for a vulnerability in the IPython notebook, reported today by Leopold Schabel on IPython's GitHub issue tracker at https://github.com/ipython/ipython/issues/6246. Email address of requester: security at ipython.org; rgbkrk at gmail.com Software name: IPython notebook Type of vulnerability: Use of insecure resources Attack outcome: Remote execution Patch/issue: https://github.com/ipython/ipython/pull/6249, https://github.com/ipython/ipython/issues/6246 Affected versions: 0.12 ? version ? 2.1 Summary: When using the IPython notebook without encryption (i.e. running the server on HTTP instead of HTTPS), mathjax is loaded over HTTP. An attacker with fortuitous network position could execute code on a local IPython notebook by modifying the mathjax javascript. This issue was fixed in the git master branch (development branch for upcoming v. 2.2) with commit cf793ebc4, on 7/31/2014: https://github.com/ipython/ipython/commit/cf793ebc4f9e8483f104667e4c73748357fa8c56 Mitigations: * Run the notebook with SSL (see http://ipython.org/ipython-doc/2/notebook/public_server.html#securing-a-notebook-server ). * Install mathjax from IPython.external.mathjax import install_mathjax install_mathjax() Regards, Kyle Kelley -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgbkrk at gmail.com Fri Aug 1 16:57:10 2014 From: rgbkrk at gmail.com (Kyle Kelley) Date: Fri, 1 Aug 2014 15:57:10 -0500 Subject: [IPython-dev] Insecure loading of mathjax Message-ID: Hey all, As reported in https://github.com/ipython/ipython/issues/6246, MathJax will load over HTTP if using the notebook with an unecncrypted connection (e.g. http://127.0.0.1:8888). Someone with an appropriate network position (the router at your local internet cafe for example) could modify the mathjax javascript before it gets to you, adding their own javascript. This would result in being able to run code on your IPython kernel with just a little bit of javascript (`IPython.notebook.kernel.execute(code)`). This issue was fixed in the git master branch (development branch for upcoming v. 2.2) with commit cf793ebc4, on 7/31/2014: https://github.com/ipython/ipython/commit/cf793ebc4f9e8483f104667e4c73748357fa8c56 Mitigations: * Run the notebook with SSL (see http://ipython.org/ipython-doc/2/notebook/public_server.html#securing-a-notebook-server). This will load MathJax over SSL via the MathJax CDN. * Install mathjax locally from IPython.external.mathjax import install_mathjax install_mathjax() We have requested a CVE for this via the oss-security mailing list. -- Kyle Kelley (@rgbkrk; http://lambdaops.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jtaylor.debian at googlemail.com Fri Aug 1 17:13:39 2014 From: jtaylor.debian at googlemail.com (Julian Taylor) Date: Fri, 01 Aug 2014 23:13:39 +0200 Subject: [IPython-dev] Insecure loading of mathjax In-Reply-To: References: Message-ID: <53DC0303.7060500@googlemail.com> On 01.08.2014 22:57, Kyle Kelley wrote: > Hey all, > > As reported in https://github.com/ipython/ipython/issues/6246, MathJax > will load over HTTP if using the notebook with an unecncrypted > connection (e.g. http://127.0.0.1:8888). Someone with an appropriate > network position (the router at your local internet cafe for example) > could modify the mathjax javascript before it gets to you, adding their > own javascript. This would result in being able to run code on your > IPython kernel with just a little bit of javascript > (`IPython.notebook.kernel.execute(code)`). > > This issue was fixed in the git master branch (development branch for > upcoming v. 2.2) with commit cf793ebc4, on 7/31/2014: > > https://github.com/ipython/ipython/commit/cf793ebc4f9e8483f104667e4c73748357fa8c56 > Fwiw, I reported this already back in version 0.12 and the Debian package has always been enforcing the local mathjax for that reason. Is the mathjax cdn certificate still a shared between all users of whatever hosting provide is behind is? Back then this was the case for the https cdn mathjax used making it quite pointless as any users of that hosting service (I think it was amazon) could serve you a forged mathjax via valid https. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From takowl at gmail.com Fri Aug 1 17:37:12 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 1 Aug 2014 14:37:12 -0700 Subject: [IPython-dev] Insecure loading of mathjax In-Reply-To: <53DC0303.7060500@googlemail.com> References: <53DC0303.7060500@googlemail.com> Message-ID: On 1 August 2014 14:13, Julian Taylor wrote: > Is the mathjax cdn certificate still a shared between all users of > whatever hosting provide is behind is? > Back then this was the case for the https cdn mathjax used making it > quite pointless as any users of that hosting service (I think it was > amazon) could serve you a forged mathjax via valid https. > Looking at the certificate details in my browser, it looks like that is still an issue. It doesn't look like it's shared between all users of the hosting service - there's a list of 30 or so domains that appear to share it. I think that means that only someone who controlled one of those domains could do a MITM attack, so it's a lot more secure than just loading it over http, but still not properly secure. Gah, what's the point of HTTPS if it gets used like this... I am very much not a security expert, so take my assessment with a large pinch of salt. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From pi at berkeley.edu Fri Aug 1 17:40:38 2014 From: pi at berkeley.edu (Paul Ivanov) Date: Fri, 1 Aug 2014 14:40:38 -0700 Subject: [IPython-dev] Insecure loading of mathjax In-Reply-To: <53DC0303.7060500@googlemail.com> References: <53DC0303.7060500@googlemail.com> Message-ID: <20140801214038.GD27567@HbI-OTOH.berkeley.edu> Hi Julian, Kyle, and list, I just wanted to publicly thank Kyle again for following through with these and ensure that they get reported and communicated in the right manner. None of the other other IPython developers have any experience with disclosing security vulnerabilities to appropriate channels, and Kyle has stepped up entirely in a volunteer capacity to do this for the benefit of the community. Thanks to you as well, Julian, for bringing that CDN certificate issue to our attention. We need all the help we can get, and I my immediate reaction to reading "...making it quite pointless..." was that Kyle is getting the stick instead of a carrot for following through and doing a better job than we would have done without him (your point about reporting this back in 0.12 is an example of our previous lack of familiarity, appreciation, and engagement with security related issues). If you have the time and interest, We'd love your help on the security side of things (contact Kyle or me offlist), and I think Kyle is striving to do a much more punctual disclosure of this vulnerability in part because of your feedback on CVE-2014-3429. I just want to make sure that we continue to have productive interactions. my sincerest appreciation to both of you, -- _ / \ A* \^ - ,./ _.`\\ / \ / ,--.S \/ \ / `"~,_ \ \ __o ? _ \<,_ /:\ --(_)/-(_)----.../ | \ --------------.......J Paul Ivanov ipython and matplotlib core developer http://pirsquared.org From takowl at gmail.com Fri Aug 1 17:55:09 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 1 Aug 2014 14:55:09 -0700 Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython Notebook on your own machine Message-ID: I've been working on a couple of tools to improve the experience of working with IPython notebooks on your own computer. First, nbopen, a tool to open a notebook in the nearest available server, or start a new server if there isn't one available. It now integrates with your file manager so you can double click on a notebook file to open it (open source desktops only for now, PRs for other systems welcome): https://github.com/takluyver/nbopen However, there's no easy way to shut down notebook servers launched by double clicking on notebooks, because there isn't a terminal where you can press ctrl-c. So I also made nbmanager, a GUI app which discovers running notebook servers and open notebooks, and lets you easily shut them down: https://github.com/takluyver/nbmanager nbopen will work on IPython 2, while nbmanager, which is more experimental, requires you to be running IPython from git at the moment. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Aug 1 17:57:45 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 1 Aug 2014 14:57:45 -0700 Subject: [IPython-dev] Insecure loading of mathjax In-Reply-To: <20140801214038.GD27567@HbI-OTOH.berkeley.edu> References: <53DC0303.7060500@googlemail.com> <20140801214038.GD27567@HbI-OTOH.berkeley.edu> Message-ID: I was about to write something along the same lines, but Paul said it much better than I. +1 to all of that... Cheers, f On Fri, Aug 1, 2014 at 2:40 PM, Paul Ivanov wrote: > Hi Julian, Kyle, and list, > > I just wanted to publicly thank Kyle again for following through > with these and ensure that they get reported and communicated in > the right manner. None of the other other IPython developers have > any experience with disclosing security vulnerabilities to > appropriate channels, and Kyle has stepped up entirely in a > volunteer capacity to do this for the benefit of the community. > > Thanks to you as well, Julian, for bringing that CDN certificate > issue to our attention. We need all the help we can get, and I > my immediate reaction to reading "...making it quite > pointless..." was that Kyle is getting the stick instead of a > carrot for following through and doing a better job than we would > have done without him (your point about reporting this back in > 0.12 is an example of our previous lack of familiarity, > appreciation, and engagement with security related issues). > > If you have the time and interest, We'd love your help on the > security side of things (contact Kyle or me offlist), and I think > Kyle is striving to do a much more punctual disclosure of this > vulnerability in part because of your feedback on CVE-2014-3429. > I just want to make sure that we continue to have productive > interactions. > > my sincerest appreciation to both of you, > -- > _ > / \ > A* \^ - > ,./ _.`\\ / \ > / ,--.S \/ \ > / `"~,_ \ \ > __o ? > _ \<,_ /:\ > --(_)/-(_)----.../ | \ > --------------.......J > Paul Ivanov > ipython and matplotlib core developer > http://pirsquared.org > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From jtaylor.debian at googlemail.com Fri Aug 1 18:03:58 2014 From: jtaylor.debian at googlemail.com (Julian Taylor) Date: Sat, 02 Aug 2014 00:03:58 +0200 Subject: [IPython-dev] Insecure loading of mathjax In-Reply-To: <20140801214038.GD27567@HbI-OTOH.berkeley.edu> References: <53DC0303.7060500@googlemail.com> <20140801214038.GD27567@HbI-OTOH.berkeley.edu> Message-ID: <53DC0ECE.6030108@googlemail.com> On 01.08.2014 23:40, Paul Ivanov wrote: > Hi Julian, Kyle, and list, > > I just wanted to publicly thank Kyle again for following through > with these and ensure that they get reported and communicated in > the right manner. None of the other other IPython developers have > any experience with disclosing security vulnerabilities to > appropriate channels, and Kyle has stepped up entirely in a > volunteer capacity to do this for the benefit of the community. > > Thanks to you as well, Julian, for bringing that CDN certificate > issue to our attention. We need all the help we can get, and I > my immediate reaction to reading "...making it quite > pointless..." was that Kyle is getting the stick instead of a > carrot for following through and doing a better job than we would > have done without him (your point about reporting this back in > 0.12 is an example of our previous lack of familiarity, > appreciation, and engagement with security related issues). I think it is great that Kyle is communicating the issues and helping fixing issues. I did not intend to give "the stick" with my statement I was referring to the situation back then, where I myself could have probably handled the situation much better. I too do not know very much about network security and how best to report issues, though I can possibly help out contacting the distribution security teams if needed. The shared certificate is probably not a huge problem as the number of AltNames for the mathjax certificate is quite small, but some googling showed that this is indeed an attack vector: http://news.netcraft.com/archives/2013/10/07/phishers-using-cloudflare-for-ssl.html But I don't know if this is relevant enough to require any more actions from IPython. > > If you have the time and interest, We'd love your help on the > security side of things (contact Kyle or me offlist), and I think > Kyle is striving to do a much more punctual disclosure of this > vulnerability in part because of your feedback on CVE-2014-3429. > I just want to make sure that we continue to have productive > interactions. > > my sincerest appreciation to both of you, > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From takowl at gmail.com Fri Aug 1 18:26:20 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 1 Aug 2014 15:26:20 -0700 Subject: [IPython-dev] Insecure loading of mathjax In-Reply-To: <53DC0ECE.6030108@googlemail.com> References: <53DC0303.7060500@googlemail.com> <20140801214038.GD27567@HbI-OTOH.berkeley.edu> <53DC0ECE.6030108@googlemail.com> Message-ID: On 1 August 2014 15:03, Julian Taylor wrote: > The shared certificate is probably not a huge problem as the number of > AltNames for the mathjax certificate is quite small, but some googling > showed that this is indeed an attack vector: > > http://news.netcraft.com/archives/2013/10/07/phishers-using-cloudflare-for-ssl.html > That doesn't appear to show someone actually MITMing an https server using the shared certificate. The concern there appears to be that cloudflare is letting people use its https certificates without much/any validation, making for more convincing phishing attacks. That wouldn't compromise a request to a specific, known URL. Amusingly, if predictably, it appears that these shared certificates are needed because of 'a lack of support for SNI in Internet Explorer on Windows XP'. I'll put this on the agenda for the next dev meeting. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From lstagner at uci.edu Fri Aug 1 18:36:46 2014 From: lstagner at uci.edu (Luke Stagner) Date: Fri, 01 Aug 2014 15:36:46 -0700 Subject: [IPython-dev] IDL/GDL kernel In-Reply-To: References: <53D99C9D.4070900@uci.edu> <53D9B7A0.5050309@uci.edu> <53D9C05D.1030204@uci.edu> Message-ID: <53DC167E.3070901@uci.edu> Thanks that did the trick. -Luke On 07/31/2014 03:55 PM, Thomas Kluyver wrote: > On 30 July 2014 21:04, Luke Stagner > wrote: > > Good news no more error. Bad news still no inline plot. I even set > it so that it reads some random png I have every time and still no go. > > > Sorry, I got the message structure wrong when I gave you an example > without trying it. This time, here's an example that I have tested and > ensured that it works: > > with open('foo.png', 'rb') as f: > > import base64 > > imgdata = base64.b64encode(f.read()) > > self.send_response(self.iopub_socket, 'display_data', > {'data':{'image/png': imgdata.decode('ascii')}}) > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From jtaylor.debian at googlemail.com Fri Aug 1 18:47:52 2014 From: jtaylor.debian at googlemail.com (Julian Taylor) Date: Sat, 02 Aug 2014 00:47:52 +0200 Subject: [IPython-dev] Insecure loading of mathjax In-Reply-To: References: <53DC0303.7060500@googlemail.com> <20140801214038.GD27567@HbI-OTOH.berkeley.edu> <53DC0ECE.6030108@googlemail.com> Message-ID: <53DC1918.60404@googlemail.com> On 02.08.2014 00:26, Thomas Kluyver wrote: > On 1 August 2014 15:03, Julian Taylor > wrote: > > The shared certificate is probably not a huge problem as the number of > AltNames for the mathjax certificate is quite small, but some googling > showed that this is indeed an attack vector: > http://news.netcraft.com/archives/2013/10/07/phishers-using-cloudflare-for-ssl.html > > > That doesn't appear to show someone actually MITMing an https server > using the shared certificate. The concern there appears to be that > cloudflare is letting people use its https certificates without much/any > validation, making for more convincing phishing attacks. That wouldn't > compromise a request to a specific, known URL. Normally if you can do an MITM and someone redirects your request mathjax.org to another domain the browser would give you a certificate error as the certificate served would not be valid for the domain you requested. In this case if someone is able to do a MITM and controls one of the domains that share the certificate with mathjax that person would be able to redirect the request to mathjax.org to his domain and serve any file. The browser would accept this as the certificate is valid for the domain even though its a different one than the request was sent to. Or would the browser complain about the change in any case? But I guess this is unlikely to be feasible/worthwhile to attack ipython users in practice. > > Amusingly, if predictably, it appears that these shared certificates are > needed because of 'a lack of support for SNI in Internet Explorer on > Windows XP'. also amusingly amazon cloudfront which (I think) mathjax used before rackspace and cloudfront now supports SNI since march this year: http://aws.amazon.com/blogs/aws/server-name-indication-sni-and-http-redirection-for-amazon-cloudfront/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From takowl at gmail.com Fri Aug 1 21:20:42 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 1 Aug 2014 18:20:42 -0700 Subject: [IPython-dev] Insecure loading of mathjax In-Reply-To: <53DC1918.60404@googlemail.com> References: <53DC0303.7060500@googlemail.com> <20140801214038.GD27567@HbI-OTOH.berkeley.edu> <53DC0ECE.6030108@googlemail.com> <53DC1918.60404@googlemail.com> Message-ID: On 1 August 2014 15:47, Julian Taylor wrote: > In this case if someone is able to do a MITM and controls one of the > domains that share the certificate with mathjax that person would be > able to redirect the request to mathjax.org to his domain and serve any > file. The browser would accept this as the certificate is valid for the > domain even though its a different one than the request was sent to. > Or would the browser complain about the change in any case? > I have opened an issue on Mathjax to work out if this is possible. It's probably most concerning for IPython, but if it's possible, it potentially affects any site loading Mathjax from the CDN. https://github.com/mathjax/MathJax/issues/885 Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgbkrk at gmail.com Fri Aug 1 21:54:29 2014 From: rgbkrk at gmail.com (Kyle Kelley) Date: Fri, 1 Aug 2014 20:54:29 -0500 Subject: [IPython-dev] Docker IPython Message-ID: Earlier this week we released an experimental base Docker image for the IPython notebook: https://registry.hub.docker.com/u/ipython/notebook/ As well as a Docker image for the notebook viewer (which will be used in production soon): https://registry.hub.docker.com/u/ipython/nbviewer/ Yay! Several folks have reached out about these images, and I'd like to get a conversation started amongst those that want to build from a common base. There are, after all, a *lot* of ipython images up on Docker already. Additionally, there is a LONG thread on the software carpentry mailing list There are some open questions from me: * Who should the user be inside the container? * Should they be able to install more packages? * In a chef cookbook and deployment setup, I just made the user named ipynb. * Is it ok to force HTTPS? * If we do, we're stuck with self signed certs or requiring them to provide them on run * This isn't really the right way to compose applications with Docker and probably makes things problematic for anyone building on top of ours * Who is our target audience? * People deploying notebooks across their organization for users? * Students (and their teachers) wanting access to an easy analytics environment? * What would you see out of a base image? Where would the value be? * Would people like to see a bleeding latest image that you can spin up to see the current state of the IPython notebook by just running `docker run ipython/notebook:bleeding-latest`? * Do people want a base image that has a large portion of the scipy stack? I'll stop with these and see where the discussion takes us. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjoern.gruening at gmail.com Sat Aug 2 02:03:38 2014 From: bjoern.gruening at gmail.com (=?UTF-8?B?QmrDtnJuIEdyw7xuaW5n?=) Date: Sat, 02 Aug 2014 08:03:38 +0200 Subject: [IPython-dev] Announcement: Galaxy IPython Integration - interactive programming in Galaxy Message-ID: <53DC7F3A.4000601@gmail.com> Hi all, we proudly present the first release of the Galaxy IPython project. Galaxy IPython is a visualization plugin which should enable Galaxy users with coding skills to easily process their data in the most flexible way. With this plugin, it is possible to analyse and post-process data without downloading datasets or entire histories. One of our aims was to make Galaxy more attractive and accessible to bioinformaticians and programmers, and we hope that this project will build some bridges. More information about Galaxy: http://galaxyproject.org/ Screencast: http://www.youtube.com/watch?v=jQDyTuYnn1k Disclaimer: Even though the Ipython notebooks can be stored and reused, this plugin will break the Galaxy philosophy of reproducibility, I feel personally bad about that, but I also think it is a great opportunity to get more bioinformaticians into Galaxy, and to get Galaxy used more often as a teaching resource. By being able to teach not only about workflows but also about data analysis tasks often necessary with Bioinformatics, Galaxy will be significantly more useful in teaching environments. A few features we have up and running: * Use IPython directly in the main window or in the Scratchbook * Completely encapsulated IPython environment with matplotlib, biopython, pandas and friends already installed. * IPython runs completely self-contained within a docker container, separate from your Galaxy data * Easy access to datasets from your current history via pre-defined IPython functions * Manipulate and plot data as you like and export your new files back into your Galaxy history * Save IPython Notebooks across analysis sessions in your Galaxy history with the click of a button. * View saved IPython Notebooks directly in HTML format, or re-open them to continue your analysis. * Self-closing and self-cleaning IPython docker container Notebooks are secure, only accessible to the intended user Please follow the installation instruction on our project page under: https://github.com/bgruening/galaxy-ipython The underlying IPython Notebook (+Galaxy sugar) is stored under: https://github.com/bgruening/docker-ipython-notebook https://registry.hub.docker.com/u/bgruening/docker-ipython-notebook/ You can also install a ipynb datatype: https://github.com/bgruening/galaxytools/tree/master/datatypes/json https://testtoolshed.g2.bx.psu.edu/view/iuc/datatyp_ipynb Comments welcome! Happy research! Eric & Bj?rn From rgbkrk at gmail.com Sat Aug 2 11:23:16 2014 From: rgbkrk at gmail.com (Kyle Kelley) Date: Sat, 2 Aug 2014 10:23:16 -0500 Subject: [IPython-dev] Announcement: Galaxy IPython Integration - interactive programming in Galaxy In-Reply-To: <53DC7F3A.4000601@gmail.com> References: <53DC7F3A.4000601@gmail.com> Message-ID: Bj?rn, Thanks for sharing. I went ahead and created a Galaxy account. What's the simplest path for trying out the IPython notebook integration within Galaxy? -- Kyle On Sat, Aug 2, 2014 at 1:03 AM, Bj?rn Gr?ning wrote: > > Hi all, > > we proudly present the first release of the Galaxy IPython project. > > Galaxy IPython is a visualization plugin which should enable Galaxy > users with coding skills to easily process their data in the most > flexible way. With this plugin, it is possible to analyse and > post-process data without downloading datasets or entire histories. One > of our aims was to make Galaxy more attractive and accessible to > bioinformaticians and programmers, and we hope that this project will > build some bridges. > > More information about Galaxy: http://galaxyproject.org/ > Screencast: http://www.youtube.com/watch?v=jQDyTuYnn1k > > Disclaimer: Even though the Ipython notebooks can be stored and reused, > this plugin will break the Galaxy philosophy of reproducibility, I feel > personally bad about that, but I also think it is a great opportunity to > get more bioinformaticians into Galaxy, and to get Galaxy used more > often as a teaching resource. By being able to teach not only about > workflows but also about data analysis tasks often necessary with > Bioinformatics, Galaxy will be significantly more useful in teaching > environments. > > A few features we have up and running: > * Use IPython directly in the main window or in the Scratchbook > * Completely encapsulated IPython environment with matplotlib, > biopython, pandas and friends already installed. > * IPython runs completely self-contained within a docker container, > separate from your Galaxy data > * Easy access to datasets from your current history via pre-defined > IPython functions > * Manipulate and plot data as you like and export your new files back > into your Galaxy history > * Save IPython Notebooks across analysis sessions in your Galaxy history > with the click of a button. > * View saved IPython Notebooks directly in HTML format, or re-open them > to continue your analysis. > * Self-closing and self-cleaning IPython docker container > Notebooks are secure, only accessible to the intended user > > > Please follow the installation instruction on our project page under: > https://github.com/bgruening/galaxy-ipython > > The underlying IPython Notebook (+Galaxy sugar) is stored under: > https://github.com/bgruening/docker-ipython-notebook > https://registry.hub.docker.com/u/bgruening/docker-ipython-notebook/ > You can also install a ipynb datatype: > https://github.com/bgruening/galaxytools/tree/master/datatypes/json > https://testtoolshed.g2.bx.psu.edu/view/iuc/datatyp_ipynb > > > Comments welcome! > Happy research! > Eric & Bj?rn > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Kyle Kelley (@rgbkrk ; http://lambdaops.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjoern.gruening at gmail.com Sat Aug 2 11:34:11 2014 From: bjoern.gruening at gmail.com (=?windows-1252?Q?Bj=F6rn_Gr=FCning?=) Date: Sat, 02 Aug 2014 17:34:11 +0200 Subject: [IPython-dev] Announcement: Galaxy IPython Integration - interactive programming in Galaxy In-Reply-To: References: <53DC7F3A.4000601@gmail.com> Message-ID: <53DD04F3.2060505@gmail.com> Hi Kyle, currently you need to have an own Galaxy installation [1]. It's relatively simple and and mainly a hg clone and ./run.sh step. If you have a running Galaxy instance you can follow the installation instruction under: https://github.com/bgruening/galaxy-ipython It's simple as: git clone https://github.com/bgruening/galaxy-ipython.git $GALAX_DIR/config/plugins/viz/ipython We are facing the problem that all our productive systems do not have access to docker, so we need a little bit of time to migrate our servers. Also, I would like to collect a few more comments and reviews before we can offer this feature to all Galaxy users. Please note, that I'm in close contact with the main Galaxy devs but I'm not part of that gang, so I have nothing to decide weather or not that feature will be added to usegalaxy.org. Thanks for your interest! Bjoern [1] https://wiki.galaxyproject.org/Admin/GetGalaxy Am 02.08.2014 um 17:23 schrieb Kyle Kelley: > Bj?rn, > > Thanks for sharing. I went ahead and created a Galaxy account. What's the > simplest path for trying out the IPython notebook integration within Galaxy? > > -- Kyle > > > On Sat, Aug 2, 2014 at 1:03 AM, Bj?rn Gr?ning > wrote: > >> >> Hi all, >> >> we proudly present the first release of the Galaxy IPython project. >> >> Galaxy IPython is a visualization plugin which should enable Galaxy >> users with coding skills to easily process their data in the most >> flexible way. With this plugin, it is possible to analyse and >> post-process data without downloading datasets or entire histories. One >> of our aims was to make Galaxy more attractive and accessible to >> bioinformaticians and programmers, and we hope that this project will >> build some bridges. >> >> More information about Galaxy: http://galaxyproject.org/ >> Screencast: http://www.youtube.com/watch?v=jQDyTuYnn1k >> >> Disclaimer: Even though the Ipython notebooks can be stored and reused, >> this plugin will break the Galaxy philosophy of reproducibility, I feel >> personally bad about that, but I also think it is a great opportunity to >> get more bioinformaticians into Galaxy, and to get Galaxy used more >> often as a teaching resource. By being able to teach not only about >> workflows but also about data analysis tasks often necessary with >> Bioinformatics, Galaxy will be significantly more useful in teaching >> environments. >> >> A few features we have up and running: >> * Use IPython directly in the main window or in the Scratchbook >> * Completely encapsulated IPython environment with matplotlib, >> biopython, pandas and friends already installed. >> * IPython runs completely self-contained within a docker container, >> separate from your Galaxy data >> * Easy access to datasets from your current history via pre-defined >> IPython functions >> * Manipulate and plot data as you like and export your new files back >> into your Galaxy history >> * Save IPython Notebooks across analysis sessions in your Galaxy history >> with the click of a button. >> * View saved IPython Notebooks directly in HTML format, or re-open them >> to continue your analysis. >> * Self-closing and self-cleaning IPython docker container >> Notebooks are secure, only accessible to the intended user >> >> >> Please follow the installation instruction on our project page under: >> https://github.com/bgruening/galaxy-ipython >> >> The underlying IPython Notebook (+Galaxy sugar) is stored under: >> https://github.com/bgruening/docker-ipython-notebook >> https://registry.hub.docker.com/u/bgruening/docker-ipython-notebook/ >> You can also install a ipynb datatype: >> https://github.com/bgruening/galaxytools/tree/master/datatypes/json >> https://testtoolshed.g2.bx.psu.edu/view/iuc/datatyp_ipynb >> >> >> Comments welcome! >> Happy research! >> Eric & Bj?rn >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From jbarratt at serialized.net Mon Aug 4 01:34:50 2014 From: jbarratt at serialized.net (Joshua Barratt) Date: Sun, 3 Aug 2014 22:34:50 -0700 Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython Notebook on your own machine In-Reply-To: References: Message-ID: Thanks for sharing those, very cool. I had been thinking about doing something like nbopen, I'm happy to see that's something I can delete off of the infinite to-do list. I created something similar to nbmanager, but much more stripped down and only allows viewing. I called mine nblist . $ nblist http://127.0.0.1:8088 | /Users/me/work/notebooks http://127.0.0.1:8089 | /Users/me/work/otherproject I also recently wrote something called nbgrep (blog post here ) which allows you to search the input cells of all your notebooks, regardless of directory. Unfortunately both of mine only work on OSX, though they could be modified to be cross-platform. Josh On Fri, Aug 1, 2014 at 2:55 PM, Thomas Kluyver wrote: > I've been working on a couple of tools to improve the experience of > working with IPython notebooks on your own computer. > > First, nbopen, a tool to open a notebook in the nearest available server, > or start a new server if there isn't one available. It now integrates with > your file manager so you can double click on a notebook file to open it > (open source desktops only for now, PRs for other systems welcome): > > https://github.com/takluyver/nbopen > > However, there's no easy way to shut down notebook servers launched by > double clicking on notebooks, because there isn't a terminal where you can > press ctrl-c. So I also made nbmanager, a GUI app which discovers running > notebook servers and open notebooks, and lets you easily shut them down: > > https://github.com/takluyver/nbmanager > > nbopen will work on IPython 2, while nbmanager, which is more > experimental, requires you to be running IPython from git at the moment. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Aug 4 01:49:02 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 3 Aug 2014 22:49:02 -0700 Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython Notebook on your own machine In-Reply-To: References: Message-ID: Neat, thanks for those. For nblist, instead of scanning processes and ports, it may be easier to use our API function list_running_servers(), especially for cross platform support: http://ipython.org/ipython-doc/dev/api/generated/IPython.html.notebookapp.html#IPython.html.notebookapp.list_running_servers nbmanager also includes a slightly higher level API around that, which is completely independent of all the GUI code (though this requires IPython 3): https://github.com/takluyver/nbmanager/blob/master/nbmanager/api.py Best wishes, Thomas On 3 August 2014 22:34, Joshua Barratt wrote: > Thanks for sharing those, very cool. I had been thinking about doing > something like nbopen, I'm happy to see that's something I can delete off > of the infinite to-do list. > > I created something similar to nbmanager, but much more stripped down and > only allows viewing. > > I called mine nblist > . > > $ nblist > http://127.0.0.1:8088 | /Users/me/work/notebooks > > http://127.0.0.1:8089 | /Users/me/work/otherproject > > > I also recently wrote something called nbgrep > (blog post here > ) > which allows you to search the input cells of all your notebooks, > regardless of directory. > > Unfortunately both of mine only work on OSX, though they could be modified > to be cross-platform. > > Josh > > On Fri, Aug 1, 2014 at 2:55 PM, Thomas Kluyver wrote: > >> I've been working on a couple of tools to improve the experience of >> working with IPython notebooks on your own computer. >> >> First, nbopen, a tool to open a notebook in the nearest available server, >> or start a new server if there isn't one available. It now integrates with >> your file manager so you can double click on a notebook file to open it >> (open source desktops only for now, PRs for other systems welcome): >> >> https://github.com/takluyver/nbopen >> >> However, there's no easy way to shut down notebook servers launched by >> double clicking on notebooks, because there isn't a terminal where you can >> press ctrl-c. So I also made nbmanager, a GUI app which discovers running >> notebook servers and open notebooks, and lets you easily shut them down: >> >> https://github.com/takluyver/nbmanager >> >> nbopen will work on IPython 2, while nbmanager, which is more >> experimental, requires you to be running IPython from git at the moment. >> >> Thomas >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbarratt at serialized.net Mon Aug 4 01:59:57 2014 From: jbarratt at serialized.net (Joshua Barratt) Date: Sun, 3 Aug 2014 22:59:57 -0700 Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython Notebook on your own machine In-Reply-To: References: Message-ID: Very cool, I looked for something like that, but somehow missed it. That's perfect and much less janky, thanks! On Sun, Aug 3, 2014 at 10:49 PM, Thomas Kluyver wrote: > Neat, thanks for those. For nblist, instead of scanning processes and > ports, it may be easier to use our API function list_running_servers(), > especially for cross platform support: > > > http://ipython.org/ipython-doc/dev/api/generated/IPython.html.notebookapp.html#IPython.html.notebookapp.list_running_servers > > nbmanager also includes a slightly higher level API around that, which is > completely independent of all the GUI code (though this requires IPython 3): > > https://github.com/takluyver/nbmanager/blob/master/nbmanager/api.py > > Best wishes, > Thomas > > > On 3 August 2014 22:34, Joshua Barratt wrote: > >> Thanks for sharing those, very cool. I had been thinking about doing >> something like nbopen, I'm happy to see that's something I can delete off >> of the infinite to-do list. >> >> I created something similar to nbmanager, but much more stripped down and >> only allows viewing. >> >> I called mine nblist >> . >> >> $ nblist >> http://127.0.0.1:8088 | /Users/me/work/notebooks >> >> http://127.0.0.1:8089 | /Users/me/work/otherproject >> >> >> I also recently wrote something called nbgrep >> (blog post here >> ) >> which allows you to search the input cells of all your notebooks, >> regardless of directory. >> >> Unfortunately both of mine only work on OSX, though they could be >> modified to be cross-platform. >> >> Josh >> >> On Fri, Aug 1, 2014 at 2:55 PM, Thomas Kluyver wrote: >> >>> I've been working on a couple of tools to improve the experience of >>> working with IPython notebooks on your own computer. >>> >>> First, nbopen, a tool to open a notebook in the nearest available >>> server, or start a new server if there isn't one available. It now >>> integrates with your file manager so you can double click on a notebook >>> file to open it (open source desktops only for now, PRs for other systems >>> welcome): >>> >>> https://github.com/takluyver/nbopen >>> >>> However, there's no easy way to shut down notebook servers launched by >>> double clicking on notebooks, because there isn't a terminal where you can >>> press ctrl-c. So I also made nbmanager, a GUI app which discovers running >>> notebook servers and open notebooks, and lets you easily shut them down: >>> >>> https://github.com/takluyver/nbmanager >>> >>> nbopen will work on IPython 2, while nbmanager, which is more >>> experimental, requires you to be running IPython from git at the moment. >>> >>> Thomas >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alessandro.gagliardi at glassdoor.com Mon Aug 4 14:18:35 2014 From: alessandro.gagliardi at glassdoor.com (Alessandro Gagliardi) Date: Mon, 4 Aug 2014 18:18:35 +0000 Subject: [IPython-dev] Download a Python-generated file over the web through IPython Notebook Message-ID: I?d like to allow users to generate a CSV in a notebook that is hosted remotely and download that file to their local machine, perhaps through a widget (or some other way that doesn?t require the command line). Is this possible? The closest thing I can think of is pandas? to_clipboard method which could then be used to paste the data into a local spreadsheet, but that seems clumsy and prone to human error. Thanks in advance! Alessandro Gagliardi | senior data scientist | Glassdoor | alessandro at glassdoor.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Aug 4 14:22:32 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 4 Aug 2014 11:22:32 -0700 Subject: [IPython-dev] Download a Python-generated file over the web through IPython Notebook In-Reply-To: References: Message-ID: On 4 August 2014 11:18, Alessandro Gagliardi < alessandro.gagliardi at glassdoor.com> wrote: > I?d like to allow users to generate a CSV in a notebook that is hosted > remotely and download that file to their local machine, perhaps through a > widget (or some other way that doesn?t require the command line). > > Is this possible? The closest thing I can think of is pandas? > to_clipboard method which could then be used to paste the data into a local > spreadsheet, but that seems clumsy and prone to human error. > It's perfectly possible so long as the kernel is running on the same machine as the notebook server (which is the default configuration). Just write the file, and then display an HTML link pointing to it using a relative URL. to_clipboard won't work, because that sets the clipboard on the machine where the code is running, not on the user's machine. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From petr.wolf at gmail.com Mon Aug 4 15:03:41 2014 From: petr.wolf at gmail.com (Petr Wolf) Date: Mon, 4 Aug 2014 15:03:41 -0400 Subject: [IPython-dev] Errors in ipython startup muted by Notebook? Message-ID: Hi, as described on http://stackoverflow.com/questions/24942378/how-to-propagate-errors-out-of-ipython-profile-to-a-notebook, I wonder how to handle errors in IPython startup code. While the IPython console prints any errors right upfront, the Notebook app seems to silence them completely. That can be confusing (as some startup commands may have been executed and others may not) and counterproductive (as the information does not seem to be logged either). What is the best way to have errors propagated out of the profile, when using nootebook? Regards, Petr -------------- next part -------------- An HTML attachment was scrubbed... URL: From ocefpaf at gmail.com Mon Aug 4 15:55:38 2014 From: ocefpaf at gmail.com (Filipe Pires Alvarenga Fernandes) Date: Mon, 4 Aug 2014 16:55:38 -0300 Subject: [IPython-dev] Automatically saving notebooks as scripts has been removed Message-ID: Hello IPython devs, I was trying the latest github checkout of the IPython source and got the following message: Automatically saving notebooks as scripts has been removed. Use `ipython nbconvert --to python [notebook]` instead. I am wondering why this had to be removed. Security? I find the automatic script saving very helpful. My work-group usually commits the `.ipynb` and the `.py` side-by-side to obtain a better diff. I know that we can workaround this with nbconvert, but doing that manually is error prone, and having a script to do that will add another layer of complexity that we did not have with the automatic conversion. Well, if the automatic conversion had to go I understand, but if there is no reason can you guys consider leaving it there? Thanks, -Filipe -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Aug 4 17:18:51 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 4 Aug 2014 14:18:51 -0700 Subject: [IPython-dev] Automatically saving notebooks as scripts has been removed In-Reply-To: References: Message-ID: I think we may have been a bit too hasty in getting rid of that. We've been talking about it for a while amongst ourselves, because it doesn't fit terribly neatly into our infrastructure, but we didn't formally deprecate it, and I don't think the alternative approaches are quite as convenient yet. I've put this in the agenda for the dev meeting on Thursday. Thanks, Thomas On 4 August 2014 12:55, Filipe Pires Alvarenga Fernandes wrote: > Hello IPython devs, > > I was trying the latest github checkout of the IPython source and got the > following message: > > Automatically saving notebooks as scripts has been removed. > Use `ipython nbconvert --to python [notebook]` instead. > > I am wondering why this had to be removed. Security? > > I find the automatic script saving very helpful. My work-group usually > commits the `.ipynb` and the `.py` side-by-side to obtain a better diff. I > know that we can workaround this with nbconvert, but doing that manually is > error prone, and having a script to do that will add another layer of > complexity that we did not have with the automatic conversion. > > Well, if the automatic conversion had to go I understand, but if there is > no reason can you guys consider leaving it there? > > Thanks, > > -Filipe > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ocefpaf at gmail.com Mon Aug 4 17:39:11 2014 From: ocefpaf at gmail.com (Filipe Pires Alvarenga Fernandes) Date: Mon, 4 Aug 2014 18:39:11 -0300 Subject: [IPython-dev] Automatically saving notebooks as scripts has been removed In-Reply-To: References: Message-ID: Got it. Thanks for looking into this. On Mon, Aug 4, 2014 at 6:18 PM, Thomas Kluyver wrote: > I think we may have been a bit too hasty in getting rid of that. We've > been talking about it for a while amongst ourselves, because it doesn't fit > terribly neatly into our infrastructure, but we didn't formally deprecate > it, and I don't think the alternative approaches are quite as convenient > yet. > > I've put this in the agenda for the dev meeting on Thursday. > > Thanks, > Thomas > > > On 4 August 2014 12:55, Filipe Pires Alvarenga Fernandes < > ocefpaf at gmail.com> wrote: > >> Hello IPython devs, >> >> I was trying the latest github checkout of the IPython source and got the >> following message: >> >> Automatically saving notebooks as scripts has been removed. >> Use `ipython nbconvert --to python [notebook]` instead. >> >> I am wondering why this had to be removed. Security? >> >> I find the automatic script saving very helpful. My work-group usually >> commits the `.ipynb` and the `.py` side-by-side to obtain a better diff. I >> know that we can workaround this with nbconvert, but doing that manually is >> error prone, and having a script to do that will add another layer of >> complexity that we did not have with the automatic conversion. >> >> Well, if the automatic conversion had to go I understand, but if there is >> no reason can you guys consider leaving it there? >> >> Thanks, >> >> -Filipe >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tylere at google.com Mon Aug 4 17:52:36 2014 From: tylere at google.com (Tyler Erickson) Date: Mon, 4 Aug 2014 14:52:36 -0700 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: Kyle, I think having an official base Docker instance will be quite useful, especially for running workshops. I added comments for my particular use case below. On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley wrote: > Earlier this week we released an experimental base Docker image for the > IPython notebook: > > https://registry.hub.docker.com/u/ipython/notebook/ > > As well as a Docker image for the notebook viewer (which will be used in > production soon): > > https://registry.hub.docker.com/u/ipython/nbviewer/ > > Yay! > > Several folks have reached out about these images, and I'd like to get a > conversation started amongst those that want to build from a common base. > There are, after all, a *lot* of ipython images up on Docker already. > Additionally, there is a LONG thread on the software carpentry mailing list > > There are some open questions from me: > > * Who should the user be inside the container? > * Should they be able to install more packages? > I don't think the user accessing the instance needs to be able to install custom packages. (It would be nice this functionality was available, but it is not critical for running workshops.) However, administrators need to have the ability to create custom images based off the docker base image that have additional packages. > * In a chef cookbook and deployment setup, I just made the user named > ipynb. > > * Is it ok to force HTTPS? > * If we do, we're stuck with self signed certs or requiring them to > provide them on run > * This isn't really the right way to compose applications with Docker > and probably makes things problematic for anyone building on top of ours > > * Who is our target audience? > * People deploying notebooks across their organization for users? > * Students (and their teachers) wanting access to an easy analytics > environment? > My interest is in providing an environment for external users of our analysis API, so it somewhat similar to the latter description. Some are in academia, but many are not. * What would you see out of a base image? Where would the value be? > > * Would people like to see a bleeding latest image that you can spin up to > see the current state of the IPython notebook by just running `docker run > ipython/notebook:bleeding-latest`? > > * Do people want a base image that has a large portion of the scipy stack? > Yes, particularly matplotlib and pandas. Cheers, Tyler I'll stop with these and see where the discussion takes us. > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy at payne.org Mon Aug 4 18:41:15 2014 From: andy at payne.org (Andrew Payne) Date: Mon, 4 Aug 2014 18:41:15 -0400 Subject: [IPython-dev] Download a Python-generated file over the web through IPython Notebook In-Reply-To: References: Message-ID: On Mon, Aug 4, 2014 at 2:18 PM, Alessandro Gagliardi < alessandro.gagliardi at glassdoor.com> wrote: > I?d like to allow users to generate a CSV in a notebook that is hosted > remotely and download that file to their local machine, perhaps through a > widget (or some other way that doesn?t require the command line). > Another approach is to use a data URI , where a link is specified "by value" (e.g. the target file contents) *vs* "by reference" where the URL points to a server. For example, this cell presents a link in the output area that downloads a small CSV file: %%html Click here to download Generalizing, you should be able to get from (say) a Pandas Dataframe to a HTML() IPython display object with a "click to download" link in a few lines of Python. Or, subclass the dataframe so that all of your dataframe objects offer a "click to download" option. Pros: - Entirely self-contained, doesn't depend on how/where IPython notebooks are served - Doesn't require kernel and web server on same machine - [90% certain] A published notebook (e.g. nbconvert) will work exactly as expected Cons: - Entire CSV content embedded in cell (may be an issue for large files) - There are corner cases of lacking or partial browser support (notably, old IE) - The file is always called "download", unless you do Javascript tricks (e.g. see: http://stackoverflow.com/questions/833015/does-execcommand-saveas-work-in-firefox/13696029#13696029 ) Also note there is a new "download" attribute in tags that makes this even easier. See: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a Unfortunately, the tag is not (yet) in all major browsers. -andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From alessandro.gagliardi at glassdoor.com Mon Aug 4 18:45:09 2014 From: alessandro.gagliardi at glassdoor.com (Alessandro Gagliardi) Date: Mon, 4 Aug 2014 22:45:09 +0000 Subject: [IPython-dev] Download a Python-generated file over the web through IPython Notebook In-Reply-To: Message-ID: Worked like a charm. Thanks! It's perfectly possible so long as the kernel is running on the same machine as the notebook server (which is the default configuration). Just write the file, and then display an HTML link pointing to it using a relative URL. to_clipboard won't work, because that sets the clipboard on the machine where the code is running, not on the user's machine. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.gibiansky at gmail.com Mon Aug 4 20:41:19 2014 From: andrew.gibiansky at gmail.com (Andrew Gibiansky) Date: Mon, 4 Aug 2014 17:41:19 -0700 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: In addition to a base image which supports IPython-for-Python, it'd be nice to have a base image for IPython-for-Other-Kernels. In my case, I would love to have an IPython docker base image that I can extend to make an IHaskell docker image. For this use case, I would *not* want the scipy stack, or in fact anything other than the core libraries required to run IPython itself. I suspect the same would go for IJulia, IRuby, and so on. -- Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgbkrk at gmail.com Tue Aug 5 01:33:12 2014 From: rgbkrk at gmail.com (Kyle Kelley) Date: Tue, 5 Aug 2014 00:33:12 -0500 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: Tyler, Taking in everyones comments and my opinion as a not-quite-sysadmin: Base image, ipython/notebook, will * have just the ipython notebook essentials * up to date (as of image build) version of pip (or conda... plead for one or the other please!) * be installed for all users (all users inside the container) * This means that any images inheriting from it can set up their own user setup, install more packages * default cmd that runs the notebook server with some default parameters * This can be overwritten by a Dockerfile that uses FROM ipython/notebook My personal take on this is that we could have an ipy user, with notebooks at /home/ipy/notebooks. Derivative image, ipython/scipystack (please suggest other names), could be * numpy * scipy * matplotlib * pandas * scikit-learn For an IHaskell base image, I'd prefer the namespace to actually be jupyter/ihaskell in the long run. There may come a time when we want multiple kernels in one container. After all, have you seen what you can do with the current master branch? Including all the kernels (and stacks) will make a hefty docker image though, so I won't progress down that path just yet. On Mon, Aug 4, 2014 at 4:52 PM, Tyler Erickson wrote: > Kyle, > > I think having an official base Docker instance will be quite useful, > especially for running workshops. > > I added comments for my particular use case below. > > On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley wrote: > >> Earlier this week we released an experimental base Docker image for the >> IPython notebook: >> >> https://registry.hub.docker.com/u/ipython/notebook/ >> >> As well as a Docker image for the notebook viewer (which will be used in >> production soon): >> >> https://registry.hub.docker.com/u/ipython/nbviewer/ >> >> Yay! >> >> Several folks have reached out about these images, and I'd like to get a >> conversation started amongst those that want to build from a common base. >> There are, after all, a *lot* of ipython images up on Docker already. >> Additionally, there is a LONG thread on the software carpentry mailing list >> >> There are some open questions from me: >> >> * Who should the user be inside the container? >> * Should they be able to install more packages? >> > > I don't think the user accessing the instance needs to be able to install > custom packages. (It would be nice this functionality was available, but it > is not critical for running workshops.) > > However, administrators need to have the ability to create custom images > based off the docker base image that have additional packages. > > >> * In a chef cookbook and deployment setup, I just made the user named >> ipynb. >> >> * Is it ok to force HTTPS? >> * If we do, we're stuck with self signed certs or requiring them to >> provide them on run >> * This isn't really the right way to compose applications with Docker >> and probably makes things problematic for anyone building on top of ours >> >> * Who is our target audience? >> * People deploying notebooks across their organization for users? >> * Students (and their teachers) wanting access to an easy analytics >> environment? >> > > My interest is in providing an environment for external users of our > analysis API, so it somewhat similar to the latter description. Some are in > academia, but many are not. > > * What would you see out of a base image? Where would the value be? >> >> * Would people like to see a bleeding latest image that you can spin up >> to see the current state of the IPython notebook by just running `docker >> run ipython/notebook:bleeding-latest`? >> >> * Do people want a base image that has a large portion of the scipy stack? >> > > Yes, particularly matplotlib and pandas. > > Cheers, > Tyler > > I'll stop with these and see where the discussion takes us. >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Kyle Kelley (@rgbkrk ; http://lambdaops.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From tylere at google.com Tue Aug 5 01:46:01 2014 From: tylere at google.com (Tyler Erickson) Date: Mon, 4 Aug 2014 22:46:01 -0700 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: On Mon, Aug 4, 2014 at 10:33 PM, Kyle Kelley wrote: > Tyler, > > Taking in everyones comments and my opinion as a not-quite-sysadmin: > > Base image, ipython/notebook, will > > * have just the ipython notebook essentials > * up to date (as of image build) version of pip (or conda... plead for one > or the other please!) > conda, please :) > * be installed for all users (all users inside the container) > * This means that any images inheriting from it can set up their own > user setup, install more packages > Sounds fine to me. > * default cmd that runs the notebook server with some default parameters > * This can be overwritten by a Dockerfile that uses FROM ipython/notebook > > My personal take on this is that we could have an ipy user, with notebooks > at /home/ipy/notebooks. > Sounds reasonable. I have no strong opinion. > Derivative image, ipython/scipystack (please suggest other names), could be > > * numpy > * scipy > * matplotlib > * pandas > * scikit-learn > I would be happy with that list. Cheers, Tyler > For an IHaskell base image, I'd prefer the namespace to actually be > jupyter/ihaskell in the long run. > > There may come a time when we want multiple kernels in one container. > After all, have you seen what you can do with the current master branch? > > > > > Including all the kernels (and stacks) will make a hefty docker image > though, so I won't progress down that path just yet. > > > > On Mon, Aug 4, 2014 at 4:52 PM, Tyler Erickson wrote: > >> Kyle, >> >> I think having an official base Docker instance will be quite useful, >> especially for running workshops. >> >> I added comments for my particular use case below. >> >> On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley wrote: >> >>> Earlier this week we released an experimental base Docker image for the >>> IPython notebook: >>> >>> https://registry.hub.docker.com/u/ipython/notebook/ >>> >>> As well as a Docker image for the notebook viewer (which will be used in >>> production soon): >>> >>> https://registry.hub.docker.com/u/ipython/nbviewer/ >>> >>> Yay! >>> >>> Several folks have reached out about these images, and I'd like to get a >>> conversation started amongst those that want to build from a common base. >>> There are, after all, a *lot* of ipython images up on Docker already. >>> Additionally, there is a LONG thread on the software carpentry mailing list >>> >>> There are some open questions from me: >>> >>> * Who should the user be inside the container? >>> * Should they be able to install more packages? >>> >> >> I don't think the user accessing the instance needs to be able to >> install custom packages. (It would be nice this functionality was >> available, but it is not critical for running workshops.) >> >> However, administrators need to have the ability to create custom images >> based off the docker base image that have additional packages. >> >> >>> * In a chef cookbook and deployment setup, I just made the user named >>> ipynb. >>> >>> * Is it ok to force HTTPS? >>> * If we do, we're stuck with self signed certs or requiring them to >>> provide them on run >>> * This isn't really the right way to compose applications with Docker >>> and probably makes things problematic for anyone building on top of ours >>> >>> * Who is our target audience? >>> * People deploying notebooks across their organization for users? >>> * Students (and their teachers) wanting access to an easy analytics >>> environment? >>> >> >> My interest is in providing an environment for external users of our >> analysis API, so it somewhat similar to the latter description. Some are in >> academia, but many are not. >> >> * What would you see out of a base image? Where would the value be? >>> >>> * Would people like to see a bleeding latest image that you can spin up >>> to see the current state of the IPython notebook by just running `docker >>> run ipython/notebook:bleeding-latest`? >>> >>> * Do people want a base image that has a large portion of the scipy >>> stack? >>> >> >> Yes, particularly matplotlib and pandas. >> >> Cheers, >> Tyler >> >> I'll stop with these and see where the discussion takes us. >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > > -- > Kyle Kelley (@rgbkrk ; http://lambdaops.com) > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Tue Aug 5 02:00:17 2014 From: wes.turner at gmail.com (Wes Turner) Date: Tue, 5 Aug 2014 01:00:17 -0500 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: On Tue, Aug 5, 2014 at 12:46 AM, Tyler Erickson wrote: > > On Mon, Aug 4, 2014 at 10:33 PM, Kyle Kelley wrote: > >> Tyler, >> >> Taking in everyones comments and my opinion as a not-quite-sysadmin: >> >> Base image, ipython/notebook, will >> >> * have just the ipython notebook essentials >> * up to date (as of image build) version of pip (or conda... plead for >> one or the other please!) >> > > conda, please :) > +1. http://conda.pydata.org/docs/build.html > > >> * be installed for all users (all users inside the container) >> * This means that any images inheriting from it can set up their own >> user setup, install more packages >> > > Sounds fine to me. > > >> * default cmd that runs the notebook server with some default parameters >> * This can be overwritten by a Dockerfile that uses FROM >> ipython/notebook >> >> My personal take on this is that we could have an ipy user, with >> notebooks at /home/ipy/notebooks. >> > > Sounds reasonable. I have no strong opinion. > > >> Derivative image, ipython/scipystack (please suggest other names), could >> be >> >> * numpy >> * scipy >> * matplotlib >> * pandas >> * scikit-learn >> > > I would be happy with that list. > * [ ] pip (which version?) * Supervisord? http://supervisord.org/ https://github.com/Supervisor/initscripts * conda https://pypi.python.org/pypi/conda * enpkg https://pypi.python.org/pypi/enstaller * pyenv https://github.com/yyuu/pyenv * tox, {... build packages } * { ad infinitum } > > Cheers, > Tyler > > > >> For an IHaskell base image, I'd prefer the namespace to actually be >> jupyter/ihaskell in the long run. >> >> There may come a time when we want multiple kernels in one container. >> After all, have you seen what you can do with the current master branch? >> >> >> >> >> Including all the kernels (and stacks) will make a hefty docker image >> though, so I won't progress down that path just yet. >> >> >> >> On Mon, Aug 4, 2014 at 4:52 PM, Tyler Erickson wrote: >> >>> Kyle, >>> >>> I think having an official base Docker instance will be quite useful, >>> especially for running workshops. >>> >>> I added comments for my particular use case below. >>> >>> On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley wrote: >>> >>>> Earlier this week we released an experimental base Docker image for the >>>> IPython notebook: >>>> >>>> https://registry.hub.docker.com/u/ipython/notebook/ >>>> >>>> As well as a Docker image for the notebook viewer (which will be used >>>> in production soon): >>>> >>>> https://registry.hub.docker.com/u/ipython/nbviewer/ >>>> >>>> Yay! >>>> >>>> Several folks have reached out about these images, and I'd like to get >>>> a conversation started amongst those that want to build from a common base. >>>> There are, after all, a *lot* of ipython images up on Docker already. >>>> Additionally, there is a LONG thread on the software carpentry mailing list >>>> >>>> There are some open questions from me: >>>> >>>> * Who should the user be inside the container? >>>> * Should they be able to install more packages? >>>> >>> >>> I don't think the user accessing the instance needs to be able to >>> install custom packages. (It would be nice this functionality was >>> available, but it is not critical for running workshops.) >>> >>> However, administrators need to have the ability to create custom images >>> based off the docker base image that have additional packages. >>> >>> >>>> * In a chef cookbook and deployment setup, I just made the user named >>>> ipynb. >>>> >>>> * Is it ok to force HTTPS? >>>> * If we do, we're stuck with self signed certs or requiring them to >>>> provide them on run >>>> * This isn't really the right way to compose applications with Docker >>>> and probably makes things problematic for anyone building on top of ours >>>> >>>> * Who is our target audience? >>>> * People deploying notebooks across their organization for users? >>>> * Students (and their teachers) wanting access to an easy analytics >>>> environment? >>>> >>> >>> My interest is in providing an environment for external users of our >>> analysis API, so it somewhat similar to the latter description. Some are in >>> academia, but many are not. >>> >>> * What would you see out of a base image? Where would the value be? >>>> >>>> * Would people like to see a bleeding latest image that you can spin up >>>> to see the current state of the IPython notebook by just running `docker >>>> run ipython/notebook:bleeding-latest`? >>>> >>>> * Do people want a base image that has a large portion of the scipy >>>> stack? >>>> >>> >>> Yes, particularly matplotlib and pandas. >>> >>> Cheers, >>> Tyler >>> >>> I'll stop with these and see where the discussion takes us. >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> >> -- >> Kyle Kelley (@rgbkrk ; http://lambdaops.com) >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgbkrk at gmail.com Tue Aug 5 11:53:24 2014 From: rgbkrk at gmail.com (Kyle Kelley) Date: Tue, 5 Aug 2014 10:53:24 -0500 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: Wes, Responses inline. On Tue, Aug 5, 2014 at 1:00 AM, Wes Turner wrote: > > On Tue, Aug 5, 2014 at 12:46 AM, Tyler Erickson wrote: > >> >> On Mon, Aug 4, 2014 at 10:33 PM, Kyle Kelley wrote: >> >>> Tyler, >>> >>> Taking in everyones comments and my opinion as a not-quite-sysadmin: >>> >>> Base image, ipython/notebook, will >>> >>> * have just the ipython notebook essentials >>> * up to date (as of image build) version of pip (or conda... plead for >>> one or the other please!) >>> >> >> conda, please :) >> > > +1. http://conda.pydata.org/docs/build.html > > >> >> >>> * be installed for all users (all users inside the container) >>> * This means that any images inheriting from it can set up their own >>> user setup, install more packages >>> >> >> Sounds fine to me. >> >> >>> * default cmd that runs the notebook server with some default >>> parameters >>> * This can be overwritten by a Dockerfile that uses FROM >>> ipython/notebook >>> >>> My personal take on this is that we could have an ipy user, with >>> notebooks at /home/ipy/notebooks. >>> >> >> Sounds reasonable. I have no strong opinion. >> >> >>> Derivative image, ipython/scipystack (please suggest other names), >>> could be >>> >>> * numpy >>> * scipy >>> * matplotlib >>> * pandas >>> * scikit-learn >>> >> >> I would be happy with that list. >> > > * [ ] pip (which version?) > > * Supervisord? http://supervisord.org/ > https://github.com/Supervisor/initscripts > Why? The process can be kept up inside the container and managed outside of the container. Does this grant something else? > > * conda https://pypi.python.org/pypi/conda > > * enpkg https://pypi.python.org/pypi/enstaller > That's just making things more complicated. Would you be able to pip install or conda install this if you really need it? > > > * pyenv https://github.com/yyuu/pyenv > No thank you. Switching kernels will be done inside the notebook itself. > > * tox, {... build packages } > tox is out of scope to me. What is your reasoning and audience? > > * { ad infinitum } > > >> >> Cheers, >> Tyler >> >> >> >>> For an IHaskell base image, I'd prefer the namespace to actually be >>> jupyter/ihaskell in the long run. >>> >>> There may come a time when we want multiple kernels in one container. >>> After all, have you seen what you can do with the current master branch? >>> >>> >>> >>> >>> Including all the kernels (and stacks) will make a hefty docker image >>> though, so I won't progress down that path just yet. >>> >>> >>> >>> On Mon, Aug 4, 2014 at 4:52 PM, Tyler Erickson >>> wrote: >>> >>>> Kyle, >>>> >>>> I think having an official base Docker instance will be quite useful, >>>> especially for running workshops. >>>> >>>> I added comments for my particular use case below. >>>> >>>> On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley wrote: >>>> >>>>> Earlier this week we released an experimental base Docker image for >>>>> the IPython notebook: >>>>> >>>>> https://registry.hub.docker.com/u/ipython/notebook/ >>>>> >>>>> As well as a Docker image for the notebook viewer (which will be used >>>>> in production soon): >>>>> >>>>> https://registry.hub.docker.com/u/ipython/nbviewer/ >>>>> >>>>> Yay! >>>>> >>>>> Several folks have reached out about these images, and I'd like to get >>>>> a conversation started amongst those that want to build from a common base. >>>>> There are, after all, a *lot* of ipython images up on Docker already. >>>>> Additionally, there is a LONG thread on the software carpentry mailing list >>>>> >>>>> There are some open questions from me: >>>>> >>>>> * Who should the user be inside the container? >>>>> * Should they be able to install more packages? >>>>> >>>> >>>> I don't think the user accessing the instance needs to be able to >>>> install custom packages. (It would be nice this functionality was >>>> available, but it is not critical for running workshops.) >>>> >>>> However, administrators need to have the ability to create custom >>>> images based off the docker base image that have additional packages. >>>> >>>> >>>>> * In a chef cookbook and deployment setup, I just made the user >>>>> named ipynb. >>>>> >>>>> * Is it ok to force HTTPS? >>>>> * If we do, we're stuck with self signed certs or requiring them to >>>>> provide them on run >>>>> * This isn't really the right way to compose applications with >>>>> Docker and probably makes things problematic for anyone building on top of >>>>> ours >>>>> >>>>> * Who is our target audience? >>>>> * People deploying notebooks across their organization for users? >>>>> * Students (and their teachers) wanting access to an easy analytics >>>>> environment? >>>>> >>>> >>>> My interest is in providing an environment for external users of our >>>> analysis API, so it somewhat similar to the latter description. Some are in >>>> academia, but many are not. >>>> >>>> * What would you see out of a base image? Where would the value be? >>>>> >>>>> * Would people like to see a bleeding latest image that you can spin >>>>> up to see the current state of the IPython notebook by just running `docker >>>>> run ipython/notebook:bleeding-latest`? >>>>> >>>>> * Do people want a base image that has a large portion of the scipy >>>>> stack? >>>>> >>>> >>>> Yes, particularly matplotlib and pandas. >>>> >>>> Cheers, >>>> Tyler >>>> >>>> I'll stop with these and see where the discussion takes us. >>>>> >>>>> _______________________________________________ >>>>> IPython-dev mailing list >>>>> IPython-dev at scipy.org >>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>>> >>> >>> >>> -- >>> Kyle Kelley (@rgbkrk ; http://lambdaops.com) >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Kyle Kelley (@rgbkrk ; http://lambdaops.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew.gibiansky at gmail.com Tue Aug 5 12:01:06 2014 From: andrew.gibiansky at gmail.com (Andrew Gibiansky) Date: Tue, 5 Aug 2014 09:01:06 -0700 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: Pip please :) Conda is fairly non-standard in the Python ecosystem in my experience; it's common for data analysis and IPython sort of things, but pip is the more common packaging standard. Though pip is less powerful than conda, it has the benefits that *every* Python developer knows its use, and I think it's much more appropriate for the base IPython notebook docker file. Again, this is from the view that I want the IPython notebook docker build as a *base* for something non-Python related; so I *just* want the bare essentials required for IPython itself, and nothing more. Conda seems like it is adding more than you really need, whereas pip does that. It may make sense to have the Python-based derived notebook (the one that includes scipy and so on) to use Conda, though. (Not sure how well pip and conda mix.) -- Andrew On Tue, Aug 5, 2014 at 8:53 AM, Kyle Kelley wrote: > Wes, > > Responses inline. > > > On Tue, Aug 5, 2014 at 1:00 AM, Wes Turner wrote: > >> >> On Tue, Aug 5, 2014 at 12:46 AM, Tyler Erickson >> wrote: >> >>> >>> On Mon, Aug 4, 2014 at 10:33 PM, Kyle Kelley wrote: >>> >>>> Tyler, >>>> >>>> Taking in everyones comments and my opinion as a not-quite-sysadmin: >>>> >>>> Base image, ipython/notebook, will >>>> >>>> * have just the ipython notebook essentials >>>> * up to date (as of image build) version of pip (or conda... plead for >>>> one or the other please!) >>>> >>> >>> conda, please :) >>> >> >> +1. http://conda.pydata.org/docs/build.html >> >> >>> >>> >>>> * be installed for all users (all users inside the container) >>>> * This means that any images inheriting from it can set up their own >>>> user setup, install more packages >>>> >>> >>> Sounds fine to me. >>> >>> >>>> * default cmd that runs the notebook server with some default >>>> parameters >>>> * This can be overwritten by a Dockerfile that uses FROM >>>> ipython/notebook >>>> >>>> My personal take on this is that we could have an ipy user, with >>>> notebooks at /home/ipy/notebooks. >>>> >>> >>> Sounds reasonable. I have no strong opinion. >>> >>> >>>> Derivative image, ipython/scipystack (please suggest other names), >>>> could be >>>> >>>> * numpy >>>> * scipy >>>> * matplotlib >>>> * pandas >>>> * scikit-learn >>>> >>> >>> I would be happy with that list. >>> >> >> * [ ] pip (which version?) >> >> * Supervisord? http://supervisord.org/ >> https://github.com/Supervisor/initscripts >> > > Why? The process can be kept up inside the container and managed outside > of the container. Does this grant something else? > > >> >> * conda https://pypi.python.org/pypi/conda >> >> * enpkg https://pypi.python.org/pypi/enstaller >> > > That's just making things more complicated. Would you be able to pip > install or conda install this if you really need it? > > >> >> >> * pyenv https://github.com/yyuu/pyenv >> > > No thank you. Switching kernels will be done inside the notebook itself. > > >> >> * tox, {... build packages } >> > > tox is out of scope to me. What is your reasoning and audience? > > >> >> * { ad infinitum } >> >> >>> >>> Cheers, >>> Tyler >>> >>> >>> >>>> For an IHaskell base image, I'd prefer the namespace to actually be >>>> jupyter/ihaskell in the long run. >>>> >>>> There may come a time when we want multiple kernels in one container. >>>> After all, have you seen what you can do with the current master branch? >>>> >>>> >>>> >>>> >>>> Including all the kernels (and stacks) will make a hefty docker image >>>> though, so I won't progress down that path just yet. >>>> >>>> >>>> >>>> On Mon, Aug 4, 2014 at 4:52 PM, Tyler Erickson >>>> wrote: >>>> >>>>> Kyle, >>>>> >>>>> I think having an official base Docker instance will be quite useful, >>>>> especially for running workshops. >>>>> >>>>> I added comments for my particular use case below. >>>>> >>>>> On Fri, Aug 1, 2014 at 6:54 PM, Kyle Kelley wrote: >>>>> >>>>>> Earlier this week we released an experimental base Docker image for >>>>>> the IPython notebook: >>>>>> >>>>>> https://registry.hub.docker.com/u/ipython/notebook/ >>>>>> >>>>>> As well as a Docker image for the notebook viewer (which will be used >>>>>> in production soon): >>>>>> >>>>>> https://registry.hub.docker.com/u/ipython/nbviewer/ >>>>>> >>>>>> Yay! >>>>>> >>>>>> Several folks have reached out about these images, and I'd like to >>>>>> get a conversation started amongst those that want to build from a common >>>>>> base. There are, after all, a *lot* of ipython images up on Docker already. >>>>>> Additionally, there is a LONG thread on the software carpentry mailing list >>>>>> >>>>>> There are some open questions from me: >>>>>> >>>>>> * Who should the user be inside the container? >>>>>> * Should they be able to install more packages? >>>>>> >>>>> >>>>> I don't think the user accessing the instance needs to be able to >>>>> install custom packages. (It would be nice this functionality was >>>>> available, but it is not critical for running workshops.) >>>>> >>>>> However, administrators need to have the ability to create custom >>>>> images based off the docker base image that have additional packages. >>>>> >>>>> >>>>>> * In a chef cookbook and deployment setup, I just made the user >>>>>> named ipynb. >>>>>> >>>>>> * Is it ok to force HTTPS? >>>>>> * If we do, we're stuck with self signed certs or requiring them to >>>>>> provide them on run >>>>>> * This isn't really the right way to compose applications with >>>>>> Docker and probably makes things problematic for anyone building on top of >>>>>> ours >>>>>> >>>>>> * Who is our target audience? >>>>>> * People deploying notebooks across their organization for users? >>>>>> * Students (and their teachers) wanting access to an easy analytics >>>>>> environment? >>>>>> >>>>> >>>>> My interest is in providing an environment for external users of our >>>>> analysis API, so it somewhat similar to the latter description. Some are in >>>>> academia, but many are not. >>>>> >>>>> * What would you see out of a base image? Where would the value be? >>>>>> >>>>>> * Would people like to see a bleeding latest image that you can spin >>>>>> up to see the current state of the IPython notebook by just running `docker >>>>>> run ipython/notebook:bleeding-latest`? >>>>>> >>>>>> * Do people want a base image that has a large portion of the scipy >>>>>> stack? >>>>>> >>>>> >>>>> Yes, particularly matplotlib and pandas. >>>>> >>>>> Cheers, >>>>> Tyler >>>>> >>>>> I'll stop with these and see where the discussion takes us. >>>>>> >>>>>> _______________________________________________ >>>>>> IPython-dev mailing list >>>>>> IPython-dev at scipy.org >>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> IPython-dev mailing list >>>>> IPython-dev at scipy.org >>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> Kyle Kelley (@rgbkrk ; http://lambdaops.com >>>> ) >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > > -- > Kyle Kelley (@rgbkrk ; http://lambdaops.com) > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsw at fnal.gov Tue Aug 5 12:11:38 2014 From: jsw at fnal.gov (Jon Wilson) Date: Tue, 5 Aug 2014 11:11:38 -0500 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: <53E1023A.7030704@fnal.gov> If any substantial fraction of your users will want scipy/numpy/matplotlib, I would (almost, see below) recommend conda. Conda was, as I understand it, created because pip left too many barriers in place against the use of scipy/numpy etc. Specifically, experience indicated that many people who might otherwise have casually investigated scientific python tools did not do so because pip required them to have a proper FORTRAN development environment set up, and they did not wish to figure out how to do this. Conda distributes binaries rather than exclusively source, which is an effective way around this sort of problem. OTOH, a pure-python package that is hosted on PyPI (and therefore installable via pip) can (usually) be trivially made into a conda package via `conda skeleton pypi `. So making pip-style packages tends to get you conda packages for almost free. pip +1 Regards, Jon On 08/05/2014 11:01 AM, Andrew Gibiansky wrote: > Pip please :) Conda is fairly non-standard in the Python ecosystem in my > experience; it's common for data analysis and IPython sort of things, > but pip is the more common packaging standard. Though pip is less > powerful than conda, it has the benefits that /every/ Python developer > knows its use, and I think it's much more appropriate for the base > IPython notebook docker file. > > Again, this is from the view that I want the IPython notebook docker > build as a /base/ for something non-Python related; so I /just/ want the > bare essentials required for IPython itself, and nothing more. Conda > seems like it is adding more than you really need, whereas pip does > that. It may make sense to have the Python-based derived notebook (the > one that includes scipy and so on) to use Conda, though. (Not sure how > well pip and conda mix.) > > -- Andrew > > From bussonniermatthias at gmail.com Tue Aug 5 13:36:02 2014 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Tue, 5 Aug 2014 19:36:02 +0200 Subject: [IPython-dev] Docker IPython In-Reply-To: <53E1023A.7030704@fnal.gov> References: <53E1023A.7030704@fnal.gov> Message-ID: <1D91D3C5-7C9C-45BD-B2C3-4127A90CC7A4@gmail.com> Le 5 ao?t 2014 ? 18:11, Jon Wilson a ?crit : > If any substantial fraction of your users will want > scipy/numpy/matplotlib, I would (almost, see below) recommend conda. Adrew?s IHaskell users will definitively not want the scipy stack, they want Haskell things. > Conda was, as I understand it, created because pip left too many > barriers in place against the use of scipy/numpy etc. Specifically, > experience indicated that many people who might otherwise have casually > investigated scientific python tools did not do so because pip required > them to have a proper FORTRAN development environment set up, and they > did not wish to figure out how to do this. > > Conda distributes binaries rather than exclusively source, which is an > effective way around this sort of problem. > > OTOH, a pure-python package that is hosted on PyPI (and therefore > installable via pip) can (usually) be trivially made into a conda > package via `conda skeleton pypi `. So making pip-style > packages tends to get you conda packages for almost free. I suppose this ? easy ? way to make PyPi package from conda package explain why continuum package are outdated by more that a year using pip :-) Even if conda seem great, I still feel sad that there is little effort to help fixing python packaging (no, replacing is not fixing), but I understand that starting from scratch might be easier. Installig SciPy was much more a pain even a few month ago than now. Keep also in mind than miniconda will be smaller if you decide to use it. I would also reming you that with 3.0, the IPython notebook can start many kernels in different languages, so nothing prevent you from installing the notebook using pip, and having a conda kernel. (it is even possible with 2.x) So +1 for pip which should be enough, or even maybe Julia Taylor PPA if compatible. ? M From rgbkrk at gmail.com Tue Aug 5 14:09:14 2014 From: rgbkrk at gmail.com (Kyle Kelley) Date: Tue, 5 Aug 2014 13:09:14 -0500 Subject: [IPython-dev] Docker IPython In-Reply-To: <1D91D3C5-7C9C-45BD-B2C3-4127A90CC7A4@gmail.com> References: <53E1023A.7030704@fnal.gov> <1D91D3C5-7C9C-45BD-B2C3-4127A90CC7A4@gmail.com> Message-ID: I'm definitely on the pip for base image camp. conda can be installed after as part of a separate image. We also don't need to manage multiple environments. It is one container after all. Derivatives can have components. On Tue, Aug 5, 2014 at 12:36 PM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > > Le 5 ao?t 2014 ? 18:11, Jon Wilson a ?crit : > > > If any substantial fraction of your users will want > > scipy/numpy/matplotlib, I would (almost, see below) recommend conda. > > Adrew?s IHaskell users will definitively not want the scipy stack, they > want Haskell things. > > > > Conda was, as I understand it, created because pip left too many > > barriers in place against the use of scipy/numpy etc. Specifically, > > experience indicated that many people who might otherwise have casually > > investigated scientific python tools did not do so because pip required > > them to have a proper FORTRAN development environment set up, and they > > did not wish to figure out how to do this. > > > > Conda distributes binaries rather than exclusively source, which is an > > effective way around this sort of problem. > > > > OTOH, a pure-python package that is hosted on PyPI (and therefore > > installable via pip) can (usually) be trivially made into a conda > > package via `conda skeleton pypi `. So making pip-style > > packages tends to get you conda packages for almost free. > > I suppose this ? easy ? way to make PyPi package from conda package explain > why continuum package are outdated by more that a year using pip :-) > > Even if conda seem great, I still feel sad that there is little effort to > help fixing > python packaging (no, replacing is not fixing), but I understand that > starting > from scratch might be easier. Installig SciPy was much more a pain even a > few > month ago than now. > > Keep also in mind than miniconda will be smaller if you decide to use it. > > I would also reming you that with 3.0, the IPython notebook can start many > kernels in different languages, > so nothing prevent you from installing the notebook using pip, and having > a conda kernel. > (it is even possible with 2.x) > > So +1 for pip which should be enough, or even maybe Julia Taylor PPA if > compatible. > ? > M > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Kyle Kelley (@rgbkrk ; http://lambdaops.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Aug 5 14:11:25 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 5 Aug 2014 11:11:25 -0700 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: On 4 August 2014 22:33, Kyle Kelley wrote: > Derivative image, ipython/scipystack (please suggest other names), could be > > * numpy > * scipy > * matplotlib > * pandas > * scikit-learn > If you call it scipystack, please put Sympy and nose in as well - a long and heated discussion last year settled on a core set of packages that distributions describing themselves as shipping the Scipy Stack should include: http://www.scipy.org/stackspec.html As for packaging systems, I'd lean towards conda - it seems more in line with our Jupyter-ish idea of cross-language tools for scientific computing. Also, while pip can now handle wheels, you can't put wheels for Linux on PyPI, so installing extension modules into a docker container with pip is still not very practical. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From tylere at google.com Tue Aug 5 14:12:19 2014 From: tylere at google.com (Tyler Erickson) Date: Tue, 5 Aug 2014 11:12:19 -0700 Subject: [IPython-dev] Docker IPython In-Reply-To: References: <53E1023A.7030704@fnal.gov> <1D91D3C5-7C9C-45BD-B2C3-4127A90CC7A4@gmail.com> Message-ID: I agree that pip is useful, because PyPI has packages that are not available in conda, and often has more recent versions. But it doesn't need to be an either/or choice, because conda plays well with pip (at least in my experience). After installing conda, I usually use it to install pip ("conda install pip") and then pick and choose from both the conda repo and PyPI. On Tue, Aug 5, 2014 at 11:09 AM, Kyle Kelley wrote: > I'm definitely on the pip for base image camp. conda can be installed > after as part of a separate image. We also don't need to manage multiple > environments. It is one container after all. Derivatives can have > components. > > > On Tue, Aug 5, 2014 at 12:36 PM, Matthias Bussonnier < > bussonniermatthias at gmail.com> wrote: > >> >> Le 5 ao?t 2014 ? 18:11, Jon Wilson a ?crit : >> >> > If any substantial fraction of your users will want >> > scipy/numpy/matplotlib, I would (almost, see below) recommend conda. >> >> Adrew?s IHaskell users will definitively not want the scipy stack, they >> want Haskell things. >> >> >> > Conda was, as I understand it, created because pip left too many >> > barriers in place against the use of scipy/numpy etc. Specifically, >> > experience indicated that many people who might otherwise have casually >> > investigated scientific python tools did not do so because pip required >> > them to have a proper FORTRAN development environment set up, and they >> > did not wish to figure out how to do this. >> > >> > Conda distributes binaries rather than exclusively source, which is an >> > effective way around this sort of problem. >> > >> > OTOH, a pure-python package that is hosted on PyPI (and therefore >> > installable via pip) can (usually) be trivially made into a conda >> > package via `conda skeleton pypi `. So making pip-style >> > packages tends to get you conda packages for almost free. >> >> I suppose this ? easy ? way to make PyPi package from conda package >> explain >> why continuum package are outdated by more that a year using pip :-) >> >> Even if conda seem great, I still feel sad that there is little effort to >> help fixing >> python packaging (no, replacing is not fixing), but I understand that >> starting >> from scratch might be easier. Installig SciPy was much more a pain even a >> few >> month ago than now. >> >> Keep also in mind than miniconda will be smaller if you decide to use it. >> >> I would also reming you that with 3.0, the IPython notebook can start >> many kernels in different languages, >> so nothing prevent you from installing the notebook using pip, and having >> a conda kernel. >> (it is even possible with 2.x) >> >> So +1 for pip which should be enough, or even maybe Julia Taylor PPA if >> compatible. >> ? >> M >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Kyle Kelley (@rgbkrk ; http://lambdaops.com) > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Aug 5 15:07:05 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 5 Aug 2014 12:07:05 -0700 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver wrote: > If you call it scipystack, please put Sympy and nose in as well - a long > and heated discussion last year settled on a core set of packages that > distributions describing themselves as shipping the Scipy Stack should > include: > > http://www.scipy.org/stackspec.html > +1. Let's use the 'scipy stack' term as per the above. I actually would vote for the 'extended' stack, that includes a few packages that in the real world lots of people do use/need, like sklearn/image. Here's a slightly outdated version of that: https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends?slide=8 Today I'd probably drop Mayav/PyTables (too specialized) but add Seaborn (lightweight, easy, plays beautifully with pandas). That would make a very compelling image out of the box, with minimal additional complexity (while there's extension code in those, none of them are harder than scipy itself). Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Tue Aug 5 16:46:56 2014 From: wes.turner at gmail.com (Wes Turner) Date: Tue, 5 Aug 2014 15:46:56 -0500 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: >> * Supervisord? http://supervisord.org/ >> https://github.com/Supervisor/initscripts > > > Why? The process can be kept up inside the container and managed outside of > the container. Does this grant something else? * platform init (init.d, systemd) * autorestart (not handled by init) * log management ("supervisorctl tail ipnb") * strip_ansi escape sequences from logs Just a suggestion. > >> >> >> * conda https://pypi.python.org/pypi/conda >> >> * enpkg https://pypi.python.org/pypi/enstaller > > > That's just making things more complicated. Would you be able to pip install > or conda install this if you really need it? One could install either conda or enpkg in addition to pip. https://us.pycon.org/2014/sponsors/ > >> >> >> >> * pyenv https://github.com/yyuu/pyenv > > > No thank you. Switching kernels will be done inside the notebook itself. Pivot_root should not be possible in userspace, over SSH or TLS. In the back of my mind, I don't know why I am implicitly adding "IPython testing" to your list of Use Cases. Is this a Python 2 or a Python 3 container? Are the OS packages still out of date? Tox also helps with multiple versions of Python. > >> >> >> * tox, {... build packages } > > > tox is out of scope to me. What is your reasoning and audience? Testing, development > >> >> >> * { ad infinitum } >> I've got a first draft of an ipython-formula for saltstack (which also depends on ZMQ and Jinja2) which should work with any sort of virtualization. Managing the TLS CA / server / browser certs with salt appears even easier than https://bitbucket.org/westurner/certtool/src/23307af8e54be08786f1d301c55d266d8a8918ec/certtool . As usual, a hard part appears to be what to install with: * system pkgs * pip packages as root * pip packages into ~/.local * pip into a ~/env virtualenv * conda into a conda environment Setting download-cache in pip.conf may be helpful. Is a "Multi-User IPython notebook server with a scipy stack" within scope for this container? * Are they all named 'ipy'? * SSH authorized_keys? * Web based SSH? * NFS / Google Drive / git-annex / Dropbox / PyFS (FUSE) mounts for notebook storage? A Docker volume? Is it easier to manage resource quotas and bursting with each user in a separate container? While log-shipping is unique in every environment, ephemeral containers present additional challenges. ... "Running a public IPython Notebook service for teaching in university courses" http://www.reddit.com/r/IPython/comments/1q3sql/running_a_public_ipython_notebook_service_for/#cdds3be From aron at ahmadia.net Tue Aug 5 17:33:27 2014 From: aron at ahmadia.net (Aron Ahmadia) Date: Tue, 5 Aug 2014 17:33:27 -0400 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: Long thread, but I'm also in favor of conda over pip for the reasons mentioned above, but primarily that it has a better chance of installing a user's needed Python dependency than pip due to its ability to install binary dependencies. A -------------- next part -------------- An HTML attachment was scrubbed... URL: From damianavila at gmail.com Tue Aug 5 17:50:47 2014 From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=) Date: Tue, 5 Aug 2014 18:50:47 -0300 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: Conflict of interest aside ;-) I think conda is a better option right now... I share the thought exposed by Thomas... 2014-08-05 18:33 GMT-03:00 Aron Ahmadia : > Long thread, but I'm also in favor of conda over pip for the reasons > mentioned above, but primarily that it has a better chance of installing a > user's needed Python dependency than pip due to its ability to install > binary dependencies. > > A > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- *Dami?n* -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Aug 5 20:38:31 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 5 Aug 2014 17:38:31 -0700 Subject: [IPython-dev] Fwd: Gateway Computing Environments workshop at SC14 - submissions due August 27 In-Reply-To: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net> References: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net> Message-ID: Are any of us going to Supercomputing this year? The science gateway people are having a workshop there, and it's a category that IPython roughly falls into, if someone wants to go and talk to them. Thomas ---------- Forwarded message ---------- From: Science Gateways Institute Date: 4 August 2014 14:22 Subject: Gateway Computing Environments workshop at SC14 - submissions due August 27 To: Thomas Kluyver Workshop call for submissions View this email in your browser Please spread the word about our *Gateway Computing Environments workshop at SC14, *http://sciencegateways.org/upcoming-events/gce14/ . *Our half-day workshop will be the morning of Friday, November 21* (nice timing for a weekend in New Orleans). Proceedings will be published in the ACM digital library and we will once again co-publish extended contributions in a special journal issue of CCPE ( http://www.cc-pe.net/journalinfo/ , in partnership with the International Workshop on Science Gateways [IWSG]). Charlie Catlett will be our keynote speaker. Charlie is doing some very interesting work in data analysis with the City of Chicago?gateways for cities, if you will?through his work as director of the Urban Center for Computation and Data https://urbanccd.org/ . We will feature lightning talks, a poster session, and a panel in addition to Charlie's talk. *Interested presenters should submit abstracts (2?4 pages) by August 27 *(submission details on our website http://sciencegateways.org/upcoming-events/gce14/ ). We invite the submission of papers related to various aspects of science gateways?their design, but also their use and impact in research and education. Though the event will be at SC, this is very definitely not just for gateways accessing supercomputers. Please help continue the tradition of fine speakers and attendees at this workshop series. You are receiving this email because you asked to be informed about the activities of the Science Gateways Institute or you have participated in an event associated with the Institute. These updates will be infrequent, so we hope that you'll continue to stay in touch with us! *Copyright ? 2014 Science Gateways Institute, All rights reserved.* *Our mailing address is:* Science Gateways Institute University of California at San Diego 9500 Gilman Dr San Diego, CA 92093-0505 Add us to your address book unsubscribe from this list update subscription preferences [image: Email Marketing Powered by MailChimp] -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.hirschfeld at gmail.com Tue Aug 5 21:40:56 2014 From: dave.hirschfeld at gmail.com (Dave Hirschfeld) Date: Wed, 6 Aug 2014 01:40:56 +0000 (UTC) Subject: [IPython-dev] Docker IPython References: Message-ID: Fernando Perez gmail.com> writes: > > > > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver gmail.com> wrote:If you call it scipystack, please put Sympy and nose in as well - a long and heated discussion last year settled on a core set of packages that distributions describing themselves as shipping the Scipy Stack should include:http://www.scipy.org/stackspec.html > > > +1. Let's use the 'scipy stack' term as per the above. > > > > > I actually would vote for the 'extended' stack, that includes a few packages that in the real world lots of people do use/need, like sklearn/image. Here's a slightly outdated version of that: > > > https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends? slide=8 > > > Today I'd probably drop Mayav/PyTables (too specialized) but add Seaborn (lightweight, easy, plays beautifully with pandas). > > That would make a very compelling image out of the box, with minimal additional complexity (while there's extension code in those, none of them are harder than scipy itself). > > > > > Cheers, > > f-- Fernando Perez ( fperez_org; http://fperez.org) PyTables could be handy as it's an optional dependency for pandas which IIUC allows you to store pandas objects in hdf5 via the HDFStore which is becoming a very common way to store pandas objects. -Dave From ellisonbg at gmail.com Tue Aug 5 23:54:21 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 5 Aug 2014 20:54:21 -0700 Subject: [IPython-dev] Fwd: Gateway Computing Environments workshop at SC14 - submissions due August 27 In-Reply-To: References: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net> Message-ID: I can't go myself, Brian On Tue, Aug 5, 2014 at 5:38 PM, Thomas Kluyver wrote: > Are any of us going to Supercomputing this year? The science gateway > people are having a workshop there, and it's a category that IPython > roughly falls into, if someone wants to go and talk to them. > > Thomas > > ---------- Forwarded message ---------- > From: Science Gateways Institute > Date: 4 August 2014 14:22 > Subject: Gateway Computing Environments workshop at SC14 - submissions due > August 27 > To: Thomas Kluyver > > > Workshop call for submissions View this email in your browser > > > Please spread the word about our *Gateway Computing Environments workshop > at SC14, *http://sciencegateways.org/upcoming-events/gce14/ > > . > > *Our half-day workshop will be the morning of Friday, November 21* (nice > timing for a weekend in New Orleans). Proceedings will be published in the > ACM digital library and we will once again co-publish extended > contributions in a special journal issue of CCPE ( > http://www.cc-pe.net/journalinfo/ > , > in partnership with the International Workshop on Science Gateways [IWSG]). > > Charlie Catlett will be our keynote speaker. Charlie is doing some very > interesting work in data analysis with the City of Chicago?gateways for > cities, if you will?through his work as director of the Urban Center for > Computation and Data https://urbanccd.org/ > > . > > We will feature lightning talks, a poster session, and a panel in addition > to Charlie's talk. *Interested presenters should submit abstracts (2?4 > pages) by August 27 *(submission details on our website > http://sciencegateways.org/upcoming-events/gce14/ > ). > We invite the submission of papers related to various aspects of science > gateways?their design, but also their use and impact in research and > education. Though the event will be at SC, this is very definitely not just > for gateways accessing supercomputers. Please help continue the tradition > of fine speakers and attendees at this workshop series. > You are receiving this email because you asked to be informed > about the activities of the Science Gateways Institute or you have > participated in an event associated with the Institute. These updates will > be infrequent, so we hope that you'll continue to stay in touch with us! > > *Copyright ? 2014 Science Gateways Institute, All rights reserved.* > > *Our mailing address is:* > Science Gateways Institute > University of California at San Diego > 9500 Gilman Dr > San Diego, CA 92093-0505 > > Add us to your address book > > > > unsubscribe from this list > > update subscription preferences > > > > [image: Email Marketing Powered by MailChimp] > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewturk at gmail.com Wed Aug 6 00:05:19 2014 From: matthewturk at gmail.com (Matthew Turk) Date: Tue, 5 Aug 2014 23:05:19 -0500 Subject: [IPython-dev] Fwd: Gateway Computing Environments workshop at SC14 - submissions due August 27 In-Reply-To: References: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net> Message-ID: For what it's worth, I think this would be a really excellent community to connect with; SciGAP and the Science Gateway conceptualization project are both really exciting projects. A few weeks ago IPython came up on their mailing list and was roundly applauded. I'll be at SC14 (the WSSSPE meeting will be great!) and while I wouldn't be a good "IPython representative," I'd be happy to serve as an "IPython advocate." Matt On Aug 5, 2014 7:40 PM, "Thomas Kluyver" wrote: > Are any of us going to Supercomputing this year? The science gateway > people are having a workshop there, and it's a category that IPython > roughly falls into, if someone wants to go and talk to them. > > Thomas > > ---------- Forwarded message ---------- > From: Science Gateways Institute > Date: 4 August 2014 14:22 > Subject: Gateway Computing Environments workshop at SC14 - submissions due > August 27 > To: Thomas Kluyver > > > Workshop call for submissions View this email in your browser > > > Please spread the word about our *Gateway Computing Environments workshop > at SC14, *http://sciencegateways.org/upcoming-events/gce14/ > > . > > *Our half-day workshop will be the morning of Friday, November 21* (nice > timing for a weekend in New Orleans). Proceedings will be published in the > ACM digital library and we will once again co-publish extended > contributions in a special journal issue of CCPE ( > http://www.cc-pe.net/journalinfo/ > , > in partnership with the International Workshop on Science Gateways [IWSG]). > > Charlie Catlett will be our keynote speaker. Charlie is doing some very > interesting work in data analysis with the City of Chicago?gateways for > cities, if you will?through his work as director of the Urban Center for > Computation and Data https://urbanccd.org/ > > . > > We will feature lightning talks, a poster session, and a panel in addition > to Charlie's talk. *Interested presenters should submit abstracts (2?4 > pages) by August 27 *(submission details on our website > http://sciencegateways.org/upcoming-events/gce14/ > ). > We invite the submission of papers related to various aspects of science > gateways?their design, but also their use and impact in research and > education. Though the event will be at SC, this is very definitely not just > for gateways accessing supercomputers. Please help continue the tradition > of fine speakers and attendees at this workshop series. > You are receiving this email because you asked to be informed > about the activities of the Science Gateways Institute or you have > participated in an event associated with the Institute. These updates will > be infrequent, so we hope that you'll continue to stay in touch with us! > > *Copyright ? 2014 Science Gateways Institute, All rights reserved.* > > *Our mailing address is:* > Science Gateways Institute > University of California at San Diego > 9500 Gilman Dr > San Diego, CA 92093-0505 > > Add us to your address book > > > > unsubscribe from this list > > update subscription preferences > > > > [image: Email Marketing Powered by MailChimp] > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From goxberry at gmail.com Wed Aug 6 00:34:34 2014 From: goxberry at gmail.com (Geoff Oxberry) Date: Tue, 5 Aug 2014 21:34:34 -0700 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: In case chiming in matters, I'm vehemently opposed to conda in a base image, because I think conda would be a great package manager if it didn't try to do too much (as of Anaconda 1.9, it was also managing Python version, and managing/creating virtualenvs). If it's in an "extended" derivative image, that's fine by me. I agree that pip is more standard, and conda is good for installing things (and only good for installing things). On Tue, Aug 5, 2014 at 6:40 PM, Dave Hirschfeld wrote: > Fernando Perez gmail.com> writes: > > > > > > > > > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver gmail.com> > wrote:If you call it scipystack, please put Sympy and nose in as well - a > long and heated discussion last year settled on a core set of packages > that distributions describing themselves as shipping the Scipy Stack > should include:http://www.scipy.org/stackspec.html > > > > > > +1. Let's use the 'scipy stack' term as per the above. > > > > > > > > > > I actually would vote for the 'extended' stack, that includes a few > packages that in the real world lots of people do use/need, like > sklearn/image. Here's a slightly outdated version of that: > > > > > > https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends? > slide=8 > > > > > > Today I'd probably drop Mayav/PyTables (too specialized) but add > Seaborn (lightweight, easy, plays beautifully with pandas). > > > > That would make a very compelling image out of the box, with minimal > additional complexity (while there's extension code in those, none of > them are harder than scipy itself). > > > > > > > > > > Cheers, > > > > f-- Fernando Perez ( fperez_org; http://fperez.org) > > PyTables could be handy as it's an optional dependency for pandas which > IIUC allows you to store pandas objects in hdf5 via the HDFStore which is > becoming a very common way to store pandas objects. > > -Dave > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Geoffrey Oxberry, Ph.D., E.I.T. goxberry at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Wed Aug 6 00:46:42 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 5 Aug 2014 21:46:42 -0700 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: I think a more useful way of approaching this would be to ask the following: what layers of docker images are useful? Here are some ideas: Base: ipython and its deps, installed using system packages or pip Base+SciPy: Base plus the extended scipy stack installed using system packages or pip Base+R kernel Base+Julia kernel ... Conda/Anaconda version: ipython and deps installed using conda. We should talk to Continuum about this Question, do docker images have ways of pulling from multiple parents? Cheers, Brian On Tue, Aug 5, 2014 at 9:34 PM, Geoff Oxberry wrote: > In case chiming in matters, I'm vehemently opposed to conda in a base image, > because I think conda would be a great package manager if it didn't try to > do too much (as of Anaconda 1.9, it was also managing Python version, and > managing/creating virtualenvs). If it's in an "extended" derivative image, > that's fine by me. I agree that pip is more standard, and conda is good for > installing things (and only good for installing things). > > > On Tue, Aug 5, 2014 at 6:40 PM, Dave Hirschfeld > wrote: >> >> Fernando Perez gmail.com> writes: >> >> > >> > >> > >> > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver gmail.com> >> wrote:If you call it scipystack, please put Sympy and nose in as well - a >> long and heated discussion last year settled on a core set of packages >> that distributions describing themselves as shipping the Scipy Stack >> should include:http://www.scipy.org/stackspec.html >> > >> > >> > +1. Let's use the 'scipy stack' term as per the above. >> > >> > >> > >> > >> > I actually would vote for the 'extended' stack, that includes a few >> packages that in the real world lots of people do use/need, like >> sklearn/image. Here's a slightly outdated version of that: >> > >> > >> > https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends? >> slide=8 >> > >> > >> > Today I'd probably drop Mayav/PyTables (too specialized) but add >> Seaborn (lightweight, easy, plays beautifully with pandas). >> > >> > That would make a very compelling image out of the box, with minimal >> additional complexity (while there's extension code in those, none of >> them are harder than scipy itself). >> > >> > >> > >> > >> > Cheers, >> > >> > f-- Fernando Perez ( fperez_org; http://fperez.org) >> >> PyTables could be handy as it's an optional dependency for pandas which >> IIUC allows you to store pandas objects in hdf5 via the HDFStore which is >> becoming a very common way to store pandas objects. >> >> -Dave >> >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > -- > Geoffrey Oxberry, Ph.D., E.I.T. > goxberry at gmail.com > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From wes.turner at gmail.com Wed Aug 6 01:24:26 2014 From: wes.turner at gmail.com (Wes Turner) Date: Wed, 6 Aug 2014 00:24:26 -0500 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: > Question, do docker images have ways of pulling from multiple parents? AFAIU, docker layers image filesystems. There is no notion of types or interfaces identity. * https://docs.docker.com/terms/layer/ * https://docs.docker.com/reference/builder/#from * https://en.wikipedia.org/wiki/Union_mount * http://www.projectatomic.io/docs/filesystems/ (a helpful overview) > Base: ipython and its deps, installed using system packages or pip * Pending a current PPA, * I'd build with *current* pip. > Base+SciPy: Base plus the extended scipy stack installed using system > packages or pip Package names: * http://www.scipy.org/stackspec.html Package versions: * http://docs.continuum.io/anaconda/pkg-docs.html * https://www.enthought.com/products/canopy/package-index/ > Base+R kernel > Base+Julia kernel AFAIK, these could be built as conda packages. .... **MathJax** Any chance we could get an `install_mathjax` in there? Something like this: https://gist.github.com/westurner/3265445#file-setup_pandas_notebook_deb-sh-L78 ... https://registry.hub.docker.com/u/ipython/notebook/dockerfile -- Wes Turner On Tue, Aug 5, 2014 at 11:46 PM, Brian Granger wrote: > I think a more useful way of approaching this would be to ask the > following: what layers of docker images are useful? Here are some > ideas: > > Base: ipython and its deps, installed using system packages or pip > Base+SciPy: Base plus the extended scipy stack installed using system > packages or pip > Base+R kernel > Base+Julia kernel > ... > Conda/Anaconda version: ipython and deps installed using conda. We > should talk to Continuum about this > > Question, do docker images have ways of pulling from multiple parents? > > Cheers, > > Brian > > On Tue, Aug 5, 2014 at 9:34 PM, Geoff Oxberry wrote: >> In case chiming in matters, I'm vehemently opposed to conda in a base image, >> because I think conda would be a great package manager if it didn't try to >> do too much (as of Anaconda 1.9, it was also managing Python version, and >> managing/creating virtualenvs). If it's in an "extended" derivative image, >> that's fine by me. I agree that pip is more standard, and conda is good for >> installing things (and only good for installing things). >> >> >> On Tue, Aug 5, 2014 at 6:40 PM, Dave Hirschfeld >> wrote: >>> >>> Fernando Perez gmail.com> writes: >>> >>> > >>> > >>> > >>> > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver gmail.com> >>> wrote:If you call it scipystack, please put Sympy and nose in as well - a >>> long and heated discussion last year settled on a core set of packages >>> that distributions describing themselves as shipping the Scipy Stack >>> should include:http://www.scipy.org/stackspec.html >>> > >>> > >>> > +1. Let's use the 'scipy stack' term as per the above. >>> > >>> > >>> > >>> > >>> > I actually would vote for the 'extended' stack, that includes a few >>> packages that in the real world lots of people do use/need, like >>> sklearn/image. Here's a slightly outdated version of that: >>> > >>> > >>> > https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends? >>> slide=8 >>> > >>> > >>> > Today I'd probably drop Mayav/PyTables (too specialized) but add >>> Seaborn (lightweight, easy, plays beautifully with pandas). >>> > >>> > That would make a very compelling image out of the box, with minimal >>> additional complexity (while there's extension code in those, none of >>> them are harder than scipy itself). >>> > >>> > >>> > >>> > >>> > Cheers, >>> > >>> > f-- Fernando Perez ( fperez_org; http://fperez.org) >>> >>> PyTables could be handy as it's an optional dependency for pandas which >>> IIUC allows you to store pandas objects in hdf5 via the HDFStore which is >>> becoming a very common way to store pandas objects. >>> >>> -Dave >>> >>> >>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> >> >> -- >> Geoffrey Oxberry, Ph.D., E.I.T. >> goxberry at gmail.com >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From tarun.gaba7 at gmail.com Wed Aug 6 02:13:22 2014 From: tarun.gaba7 at gmail.com (TARUN GABA) Date: Wed, 6 Aug 2014 11:43:22 +0530 Subject: [IPython-dev] Expected date for next release Message-ID: Hi What is the expected date for the next IPython release? Are the release dates available on web beforehand, or are they decided some time before the actual release? I am working on a package which has IPython (dev) as a dependency, and I want to base the next release of the package in accordance with the IPython release. Thanks Tarun Gaba -------------- next part -------------- An HTML attachment was scrubbed... URL: From rgbkrk at gmail.com Wed Aug 6 02:36:16 2014 From: rgbkrk at gmail.com (Kyle Kelley) Date: Wed, 6 Aug 2014 01:36:16 -0500 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: Brian, that is right in line with what I'd like to see. +1000 > Question, do docker images have ways of pulling from multiple parents? Docker images can only have a single parent. On Tue, Aug 5, 2014 at 11:46 PM, Brian Granger wrote: > I think a more useful way of approaching this would be to ask the > following: what layers of docker images are useful? Here are some > ideas: > > Base: ipython and its deps, installed using system packages or pip > Base+SciPy: Base plus the extended scipy stack installed using system > packages or pip > Base+R kernel > Base+Julia kernel > ... > Conda/Anaconda version: ipython and deps installed using conda. We > should talk to Continuum about this > > Question, do docker images have ways of pulling from multiple parents? > > Cheers, > > Brian > > On Tue, Aug 5, 2014 at 9:34 PM, Geoff Oxberry wrote: > > In case chiming in matters, I'm vehemently opposed to conda in a base > image, > > because I think conda would be a great package manager if it didn't try > to > > do too much (as of Anaconda 1.9, it was also managing Python version, and > > managing/creating virtualenvs). If it's in an "extended" derivative > image, > > that's fine by me. I agree that pip is more standard, and conda is good > for > > installing things (and only good for installing things). > > > > > > On Tue, Aug 5, 2014 at 6:40 PM, Dave Hirschfeld < > dave.hirschfeld at gmail.com> > > wrote: > >> > >> Fernando Perez gmail.com> writes: > >> > >> > > >> > > >> > > >> > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver > gmail.com> > >> wrote:If you call it scipystack, please put Sympy and nose in as well - > a > >> long and heated discussion last year settled on a core set of packages > >> that distributions describing themselves as shipping the Scipy Stack > >> should include:http://www.scipy.org/stackspec.html > >> > > >> > > >> > +1. Let's use the 'scipy stack' term as per the above. > >> > > >> > > >> > > >> > > >> > I actually would vote for the 'extended' stack, that includes a few > >> packages that in the real world lots of people do use/need, like > >> sklearn/image. Here's a slightly outdated version of that: > >> > > >> > > >> > > https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends? > >> slide=8 > >> > > >> > > >> > Today I'd probably drop Mayav/PyTables (too specialized) but add > >> Seaborn (lightweight, easy, plays beautifully with pandas). > >> > > >> > That would make a very compelling image out of the box, with minimal > >> additional complexity (while there's extension code in those, none of > >> them are harder than scipy itself). > >> > > >> > > >> > > >> > > >> > Cheers, > >> > > >> > f-- Fernando Perez ( fperez_org; http://fperez.org) > >> > >> PyTables could be handy as it's an optional dependency for pandas which > >> IIUC allows you to store pandas objects in hdf5 via the HDFStore which > is > >> becoming a very common way to store pandas objects. > >> > >> -Dave > >> > >> > >> > >> > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > > > -- > > Geoffrey Oxberry, Ph.D., E.I.T. > > goxberry at gmail.com > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Kyle Kelley (@rgbkrk ; http://lambdaops.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjoern.gruening at gmail.com Wed Aug 6 12:03:03 2014 From: bjoern.gruening at gmail.com (bjoern.gruening@googlemail.com) Date: Wed, 6 Aug 2014 18:03:03 +0200 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: Hi, as one of the IPython-Galaxy [1] developers we had such a similar discussion about our IPython-docker image. I really think we need to define you target audience. Imho, for an IPython base Image our target audience it quite small. It should deliver a stable and maintained IPython container. So everyone can build on-top of it. Later we can build a second container, derived from it, with a full installed scientific stack for teaching and so on ... If you include to many tools it will not be used by many developers as base image because to many peoples will disagree with your decision. I would vote for a real basic set ... even the scipy stack is really questionable for a "base image". If we go with the scipy stack than I would reference the scipy community: http://www.scipy.org/stackspec.html Keep it small but stable, care about updates and documentation and let other grow on top of your image. Cheers, Bjoern [1] https://github.com/bgruening/galaxy-ipython 2014-08-06 8:36 GMT+02:00 Kyle Kelley : > Brian, that is right in line with what I'd like to see. +1000 > > > Question, do docker images have ways of pulling from multiple parents? > > Docker images can only have a single parent. > > > On Tue, Aug 5, 2014 at 11:46 PM, Brian Granger > wrote: > >> I think a more useful way of approaching this would be to ask the >> following: what layers of docker images are useful? Here are some >> ideas: >> >> Base: ipython and its deps, installed using system packages or pip >> Base+SciPy: Base plus the extended scipy stack installed using system >> packages or pip >> Base+R kernel >> Base+Julia kernel >> ... >> Conda/Anaconda version: ipython and deps installed using conda. We >> should talk to Continuum about this >> >> Question, do docker images have ways of pulling from multiple parents? >> >> Cheers, >> >> Brian >> >> On Tue, Aug 5, 2014 at 9:34 PM, Geoff Oxberry wrote: >> > In case chiming in matters, I'm vehemently opposed to conda in a base >> image, >> > because I think conda would be a great package manager if it didn't try >> to >> > do too much (as of Anaconda 1.9, it was also managing Python version, >> and >> > managing/creating virtualenvs). If it's in an "extended" derivative >> image, >> > that's fine by me. I agree that pip is more standard, and conda is good >> for >> > installing things (and only good for installing things). >> > >> > >> > On Tue, Aug 5, 2014 at 6:40 PM, Dave Hirschfeld < >> dave.hirschfeld at gmail.com> >> > wrote: >> >> >> >> Fernando Perez gmail.com> writes: >> >> >> >> > >> >> > >> >> > >> >> > On Tue, Aug 5, 2014 at 11:11 AM, Thomas Kluyver >> gmail.com> >> >> wrote:If you call it scipystack, please put Sympy and nose in as well >> - a >> >> long and heated discussion last year settled on a core set of packages >> >> that distributions describing themselves as shipping the Scipy Stack >> >> should include:http://www.scipy.org/stackspec.html >> >> > >> >> > >> >> > +1. Let's use the 'scipy stack' term as per the above. >> >> > >> >> > >> >> > >> >> > >> >> > I actually would vote for the 'extended' stack, that includes a few >> >> packages that in the real world lots of people do use/need, like >> >> sklearn/image. Here's a slightly outdated version of that: >> >> > >> >> > >> >> > >> https://speakerdeck.com/fperez/pydata-2013-keynote-ipython-and-friends? >> >> slide=8 >> >> > >> >> > >> >> > Today I'd probably drop Mayav/PyTables (too specialized) but add >> >> Seaborn (lightweight, easy, plays beautifully with pandas). >> >> > >> >> > That would make a very compelling image out of the box, with minimal >> >> additional complexity (while there's extension code in those, none of >> >> them are harder than scipy itself). >> >> > >> >> > >> >> > >> >> > >> >> > Cheers, >> >> > >> >> > f-- Fernando Perez ( fperez_org; http://fperez.org) >> >> >> >> PyTables could be handy as it's an optional dependency for pandas which >> >> IIUC allows you to store pandas objects in hdf5 via the HDFStore which >> is >> >> becoming a very common way to store pandas objects. >> >> >> >> -Dave >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> IPython-dev mailing list >> >> IPython-dev at scipy.org >> >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> > >> > >> > >> > -- >> > Geoffrey Oxberry, Ph.D., E.I.T. >> > goxberry at gmail.com >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> >> >> >> -- >> Brian E. Granger >> Cal Poly State University, San Luis Obispo >> @ellisonbg on Twitter and GitHub >> bgranger at calpoly.edu and ellisonbg at gmail.com >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Kyle Kelley (@rgbkrk ; http://lambdaops.com) > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Aug 6 13:06:06 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 6 Aug 2014 10:06:06 -0700 Subject: [IPython-dev] Expected date for next release In-Reply-To: References: Message-ID: Hi Tarun, On 5 August 2014 23:13, TARUN GABA wrote: > What is the expected date for the next IPython release? > Are the release dates available on web beforehand, or are they decided > some time before the actual release? > > I am working on a package which has IPython (dev) as a dependency, > and I want to base the next release of the package in accordance > with the IPython release. > The next major release will be some time this autumn. We try to put these on our roadmap, although it's not always up to date: https://github.com/ipython/ipython/wiki/Roadmap:-IPython Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Aug 6 14:18:19 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 6 Aug 2014 11:18:19 -0700 Subject: [IPython-dev] Docker IPython In-Reply-To: References: Message-ID: On Tue, Aug 5, 2014 at 9:46 PM, Brian Granger wrote: > Base: ipython and its deps, installed using system packages or pip > Base+SciPy: Base plus the extended scipy stack installed using system > packages or pip > Base+R kernel > Base+Julia kernel > ... > Conda/Anaconda version: ipython and deps installed using conda. We > should talk to Continuum about this > +1 And my vote is that we (ipython/jupyter) only deal with the first one or two at the most. We should offer an official base image, possibly with the scipy stack (with apt/pip), and let the rest of the community work on the derivative ones (by community I mean both folks like R/Julia/Galaxy teams and Continuum/Enthought/any other company). We can't over-stretch ourselves too much, and we want a good ecosystem of well-targeted images. But it's useful to have a base one we maintain, if nothing else so that we understand what problems may be common to all these scenarios and we solve them once and correctly. Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Aug 6 14:20:47 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 6 Aug 2014 11:20:47 -0700 Subject: [IPython-dev] Fwd: Gateway Computing Environments workshop at SC14 - submissions due August 27 In-Reply-To: References: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net> Message-ID: On Tue, Aug 5, 2014 at 9:05 PM, Matthew Turk wrote: > I'll be at SC14 (the WSSSPE meeting will be great!) and while I wouldn't > be a good "IPython representative," I'd be happy to serve as an "IPython > advocate." > > We can deputize you ;) I think you know the project well enough that I'm sure you can do an excellent job conveying what part of the problem space we try to address. Obviously don't feel pressured into doing yet one more job, but it would be great if you can pitch in when questions about IPython arise, and send any further discussions our way. I can't go this year either, though I probably should. It's a near-certainty that I'll attend SC'15 though. Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthewturk at gmail.com Wed Aug 6 14:33:17 2014 From: matthewturk at gmail.com (Matthew Turk) Date: Wed, 6 Aug 2014 13:33:17 -0500 Subject: [IPython-dev] Fwd: Gateway Computing Environments workshop at SC14 - submissions due August 27 In-Reply-To: References: <52c619f5272025e56bf9829f53b55774f99.20140804212218@mail66.atl91.mcsv.net> Message-ID: On Wed, Aug 6, 2014 at 1:20 PM, Fernando Perez wrote: > > On Tue, Aug 5, 2014 at 9:05 PM, Matthew Turk wrote: >> >> I'll be at SC14 (the WSSSPE meeting will be great!) and while I wouldn't >> be a good "IPython representative," I'd be happy to serve as an "IPython >> advocate." > > We can deputize you ;) I think you know the project well enough that I'm > sure you can do an excellent job conveying what part of the problem space we > try to address. Obviously don't feel pressured into doing yet one more job, > but it would be great if you can pitch in when questions about IPython > arise, and send any further discussions our way. > Sounds good! > I can't go this year either, though I probably should. It's a near-certainty > that I'll attend SC'15 though. > > Cheers, > > f > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From benjaminrk at gmail.com Wed Aug 6 17:08:58 2014 From: benjaminrk at gmail.com (MinRK) Date: Wed, 6 Aug 2014 14:08:58 -0700 Subject: [IPython-dev] Automatically saving notebooks as scripts has been removed In-Reply-To: References: Message-ID: The main reason it is removed is that nbformat had a notion of ?save as .py?, which is removed in IPython 3.0 because this functionality has moved to nbconvert in IPython 1.0. We could, of course, let the appropriate nbconvert --to python call be implied by --script instead of the removed Python API, but I don?t think --script was a good idea in the first place. So it?s not a question of whether to use nbconvert or not - you have to use nbconvert to get .py files from .ipynb. It?s just a question of whether you can *automatically* use nbconvert on save via --script. I think if we want to support this kind of thing, more generic save/rename hooks are better than a single special-case flag. -MinRK ? On Mon, Aug 4, 2014 at 2:39 PM, Filipe Pires Alvarenga Fernandes < ocefpaf at gmail.com> wrote: > Got it. Thanks for looking into this. > > > > On Mon, Aug 4, 2014 at 6:18 PM, Thomas Kluyver wrote: > >> I think we may have been a bit too hasty in getting rid of that. We've >> been talking about it for a while amongst ourselves, because it doesn't fit >> terribly neatly into our infrastructure, but we didn't formally deprecate >> it, and I don't think the alternative approaches are quite as convenient >> yet. >> >> I've put this in the agenda for the dev meeting on Thursday. >> >> Thanks, >> Thomas >> >> >> On 4 August 2014 12:55, Filipe Pires Alvarenga Fernandes < >> ocefpaf at gmail.com> wrote: >> >>> Hello IPython devs, >>> >>> I was trying the latest github checkout of the IPython source and got >>> the following message: >>> >>> Automatically saving notebooks as scripts has been removed. >>> Use `ipython nbconvert --to python [notebook]` instead. >>> >>> I am wondering why this had to be removed. Security? >>> >>> I find the automatic script saving very helpful. My work-group usually >>> commits the `.ipynb` and the `.py` side-by-side to obtain a better diff. I >>> know that we can workaround this with nbconvert, but doing that manually is >>> error prone, and having a script to do that will add another layer of >>> complexity that we did not have with the automatic conversion. >>> >>> Well, if the automatic conversion had to go I understand, but if there >>> is no reason can you guys consider leaving it there? >>> >>> Thanks, >>> >>> -Filipe >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Wed Aug 6 17:11:36 2014 From: benjaminrk at gmail.com (MinRK) Date: Wed, 6 Aug 2014 14:11:36 -0700 Subject: [IPython-dev] Errors in ipython startup muted by Notebook? In-Reply-To: References: Message-ID: Yes, output from the IPython kernel is associated with the input that caused it via a notion of a parent. In the notebook, each cell causes a particular set of output. If output is received and not associated with a cell on the page, it is discarded. Output from startup files has no parent, and is thus discarded by the notebook. If we had a place to display orphaned output, we could do this. We could also consider sending output to sys.__stderr__ when parent=None, which would mean that kernel output with no parent would at least go to the terminal in which the kernel was started. -MinRK ? On Mon, Aug 4, 2014 at 12:03 PM, Petr Wolf wrote: > Hi, > > as described on > http://stackoverflow.com/questions/24942378/how-to-propagate-errors-out-of-ipython-profile-to-a-notebook, > I wonder how to handle errors in IPython startup code. > > While the IPython console prints any errors right upfront, the Notebook > app seems to silence them completely. > > That can be confusing (as some startup commands may have been executed and > others may not) and counterproductive (as the information does not seem to > be logged either). > > What is the best way to have errors propagated out of the profile, when > using nootebook? > > Regards, > Petr > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Aug 6 17:16:34 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 6 Aug 2014 14:16:34 -0700 Subject: [IPython-dev] Errors in ipython startup muted by Notebook? In-Reply-To: References: Message-ID: On 6 August 2014 14:11, MinRK wrote: > We could also consider sending output to sys.__stderr__ when parent=None, > which would mean that kernel output with no parent would at least go to the > terminal in which the kernel was started. > Though we should also consider that not every kernel is launched from a visible terminal - users on the multiuser server won't be starting from a terminal, and now that I can double click on notebooks to open them, I'm often running the notebook without a terminal as well. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Aug 6 17:47:17 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 6 Aug 2014 14:47:17 -0700 Subject: [IPython-dev] Errors in ipython startup muted by Notebook? In-Reply-To: References: Message-ID: On Wed, Aug 6, 2014 at 2:16 PM, Thomas Kluyver wrote: > Though we should also consider that not every kernel is launched from a > visible terminal - users on the multiuser server won't be starting from a > terminal, and now that I can double click on notebooks to open them, I'm > often running the notebook without a terminal as well. > Can you see what these give in one of your fully headless kernels? n [9]: sys.__stderr__.fileno() Out[9]: 2 In [10]: sys.__stderr__.isatty() Out[10]: True In [11]: sys.__stderr__.closed Out[11]: False We should be able to detect when stderr isn't meaningfully attached to anything, and at least redirect orphaned output to a dump log... Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Aug 6 17:49:10 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 6 Aug 2014 14:49:10 -0700 Subject: [IPython-dev] Insecure loading of mathjax In-Reply-To: References: <53DC0303.7060500@googlemail.com> <20140801214038.GD27567@HbI-OTOH.berkeley.edu> <53DC0ECE.6030108@googlemail.com> <53DC1918.60404@googlemail.com> Message-ID: My discussions with Mathjax developers led me to post a question on security.stackexchange, and I am now satisfied that what Cloudflare is doing with SSL certificates, although it is somewhat out of the ordinary, does not allow someone controlling one of those sites to MITM requests to another of them. So if we load Mathjax over HTTPS, we are only trusting mathjax.org and Cloudflare. https://github.com/mathjax/MathJax/issues/885 http://security.stackexchange.com/q/64738/53098 Thanks, Thomas On 1 August 2014 18:20, Thomas Kluyver wrote: > On 1 August 2014 15:47, Julian Taylor > wrote: > >> In this case if someone is able to do a MITM and controls one of the >> domains that share the certificate with mathjax that person would be >> able to redirect the request to mathjax.org to his domain and serve any >> file. The browser would accept this as the certificate is valid for the >> domain even though its a different one than the request was sent to. >> Or would the browser complain about the change in any case? >> > > I have opened an issue on Mathjax to work out if this is possible. It's > probably most concerning for IPython, but if it's possible, it potentially > affects any site loading Mathjax from the CDN. > > https://github.com/mathjax/MathJax/issues/885 > > Thomas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Aug 6 17:56:24 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 6 Aug 2014 14:56:24 -0700 Subject: [IPython-dev] Insecure loading of mathjax In-Reply-To: References: <53DC0303.7060500@googlemail.com> <20140801214038.GD27567@HbI-OTOH.berkeley.edu> <53DC0ECE.6030108@googlemail.com> <53DC1918.60404@googlemail.com> Message-ID: On Wed, Aug 6, 2014 at 2:49 PM, Thomas Kluyver wrote: > My discussions with Mathjax developers led me to post a question on > security.stackexchange, and I am now satisfied that what Cloudflare is > doing with SSL certificates, although it is somewhat out of the ordinary, > does not allow someone controlling one of those sites to MITM requests to > another of them. So if we load Mathjax over HTTPS, we are only trusting > mathjax.org and Cloudflare. > > https://github.com/mathjax/MathJax/issues/885 > http://security.stackexchange.com/q/64738/53098 > Excellent and very informative. Thomas, many thanks for following this one through to the end. Glad it's not a problem, but it was very useful to understand exactly how/why. Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Aug 6 19:25:16 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 6 Aug 2014 16:25:16 -0700 Subject: [IPython-dev] IPython 2.2 release, including security fix Message-ID: Hi all, We've just released IPython 2.2, which rolls up a few bugfixes that have been backported since the last release. This includes a fix for a security issue posted a few days ago, which could have allowed an attacker who could intercept network traffic (e.g. on insecure wifi) to intercept a request for Mathjax and send back malicious javascript which would run in the notebook. The fix was simply to always load mathjax over HTTPS: https://github.com/ipython/ipython/pull/6249 You can see the full list of issues and PRs for this release here: http://ipython.org/ipython-doc/stable/whatsnew/github-stats-2.0.html#issues-closed-in-2-2-0 You can install this release in any of the usual ways. http://ipython.org/install.html Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Aug 6 19:32:54 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 6 Aug 2014 16:32:54 -0700 Subject: [IPython-dev] IPython 2.2 release, including security fix In-Reply-To: References: Message-ID: Thanks everyone!! On Wed, Aug 6, 2014 at 4:25 PM, Thomas Kluyver wrote: > Hi all, > > We've just released IPython 2.2, which rolls up a few bugfixes that have > been backported since the last release. This includes a fix for a security > issue posted a few days ago, which could have allowed an attacker who could > intercept network traffic (e.g. on insecure wifi) to intercept a request > for Mathjax and send back malicious javascript which would run in the > notebook. The fix was simply to always load mathjax over HTTPS: > https://github.com/ipython/ipython/pull/6249 > > You can see the full list of issues and PRs for this release here: > > http://ipython.org/ipython-doc/stable/whatsnew/github-stats-2.0.html#issues-closed-in-2-2-0 > > You can install this release in any of the usual ways. > http://ipython.org/install.html > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Aug 6 19:37:54 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 6 Aug 2014 16:37:54 -0700 Subject: [IPython-dev] Errors in ipython startup muted by Notebook? In-Reply-To: References: Message-ID: On 6 August 2014 14:47, Fernando Perez wrote: > Can you see what these give in one of your fully headless kernels? In [6]: sys.__stderr__.fileno() Out[6]: 2 In [7]: sys.__stderr__.isatty() Out[7]: False In [8]: sys.__stderr__.closed Out[8]: False So the difference is that it's not a TTY, which makes sense. I guess the process launcher pipes stdout and stderr to a log file somewhere, or just to /dev/null. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Aug 6 20:07:38 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 6 Aug 2014 17:07:38 -0700 Subject: [IPython-dev] Errors in ipython startup muted by Notebook? In-Reply-To: References: Message-ID: On Wed, Aug 6, 2014 at 4:37 PM, Thomas Kluyver wrote: > So the difference is that it's not a TTY, which makes sense. I guess the > process launcher pipes stdout and stderr to a log file somewhere, or just > to /dev/null. > OK, I've tested and isatty() returns True for a normally started notebook and kernel (opened from a terminal). So that could be the way to test, and in that case, to use a file-based logger for all orphaned output. Otherwise, debugging this stuff will be insanely hard :) -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Aug 6 21:37:38 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 6 Aug 2014 18:37:38 -0700 Subject: [IPython-dev] Announcement: Galaxy IPython Integration - interactive programming in Galaxy In-Reply-To: <53DC7F3A.4000601@gmail.com> References: <53DC7F3A.4000601@gmail.com> Message-ID: Hi Bjorn, first, many thanks for posting this! On Fri, Aug 1, 2014 at 11:03 PM, Bj?rn Gr?ning wrote: > Disclaimer: Even though the Ipython notebooks can be stored and reused, > this plugin will break the Galaxy philosophy of reproducibility, I feel > personally bad about that, > Could you unpack that statement a little bit? We think of IPython as a tool that can *help* reproducible research, so I'd really like to understand better what you mean by the above... Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjoern.gruening at gmail.com Thu Aug 7 02:56:54 2014 From: bjoern.gruening at gmail.com (=?windows-1252?Q?Bj=F6rn_Gr=FCning?=) Date: Thu, 07 Aug 2014 08:56:54 +0200 Subject: [IPython-dev] Announcement: Galaxy IPython Integration - interactive programming in Galaxy In-Reply-To: References: <53DC7F3A.4000601@gmail.com> Message-ID: <53E32336.3010405@gmail.com> Hi Fernando, >> Disclaimer: Even though the Ipython notebooks can be stored and reused, >> this plugin will break the Galaxy philosophy of reproducibility, I feel >> personally bad about that, >> > > Could you unpack that statement a little bit? We think of IPython as a tool > that can *help* reproducible research, so I'd really like to understand > better what you mean by the above... In Galaxy land we are kind of extreme about reproducibility, which is good. With the IPython container we loose a little bit of this, but we gain a lot of flexibility for developers. For example in Galaxy every dataset and tool is versioned. If you process your data with matplotlib-1.2.1 we are logging this and you can rerun your analysis with the same settings, versions and so on ... If we are now using the IPython extension, we don't have such a fine grained support for packages. We could version the containers and restrict the container to install packages ... but I feel like the status quo is acceptable, since I assume our users are mostly bioinformaticians. But ... we still maintain the ipynb file. The user can, at any time, save the file to the history, we will attach the ipynb as metadata to every dataset that is exported. So that you can use the Galaxy reload button and you will be back in IPython with all your code. I think this is already a high level of reproducibility and I'm whining on a high level :) Cheers, Bjoern > Cheers > > f > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From kb201a at gmail.com Thu Aug 7 09:16:56 2014 From: kb201a at gmail.com (Keith) Date: Thu, 7 Aug 2014 09:16:56 -0400 Subject: [IPython-dev] tornado app Message-ID: <20140807131656.GN24532@hike.sph.harvard.edu> Hi, I have an existing tornado application and am thinking about leveraging (moving it into) ipython notebook. Currently the app defines its own request handlers and websocket for communication with the client browser. I was thinking that the app could generate a notebook dynamically which would be deployed on the client's browser. Then communication to the server could take place within the notebook's cells. Assuming this is possible (I'm new to ipython), are there hooks in ipython notebook that would allow me to extend tornado i.e. add my own routes and request handlers etc? thanks, Keith From paddy at paddymullen.com Thu Aug 7 11:17:53 2014 From: paddy at paddymullen.com (Paddy Mullen) Date: Thu, 7 Aug 2014 11:17:53 -0400 Subject: [IPython-dev] Url routes for individual cells Message-ID: I would like to be able to "pop out" analysis cells into separate windows. It would be useful if each cell could get it's own url route. with routes like http://localhost:8888/notebooks/analysis.ipynb /cells/23 implementing the pop out feature would be easy. I think it would help to have a concept of visual cell number vs executed cell number. I don't think the notebook has this right now. It would be annoying to have to pop out a new window every time you re execute the original code cell. I saw this pull request, it's not clear if this is doing something similar https://github.com/ipython/ipython/pull/6148 IFrames in all places. Any thoughts on this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From paddy at paddymullen.com Thu Aug 7 11:24:53 2014 From: paddy at paddymullen.com (Paddy Mullen) Date: Thu, 7 Aug 2014 11:24:53 -0400 Subject: [IPython-dev] Url route to download a notebook and open it Message-ID: It would be useful to have an url route that downloaded a notebook and opened it. I could see a route like http://localhost:8888/download/ https://raw.githubusercontent.com/ipython/ipython/master/examples/Notebook/SymPy.ipynb that would download the SymPy notebook and open it. This would make it easy to link to generated notebooks. The alternative workflow now is to provide a download link, and have the user drag that file into the notebook filebrowser. I realize that this would be a security risk, so it would probably be a feature that was best disabled by default. A configurable whitelist of allowable download domains could help a lot. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paddy at paddymullen.com Thu Aug 7 11:36:33 2014 From: paddy at paddymullen.com (Paddy Mullen) Date: Thu, 7 Aug 2014 11:36:33 -0400 Subject: [IPython-dev] API to execute cells of notebook Message-ID: I have played with the new widgets in the IPython notebook and it's a very useful tool. One thing that I would like to do is have the top cell of a notebook contain widgets that setup global variables, and have the rest of the cells function as a report based on the global variables. My idea is that there would be a "Run it" button that executes the rest of the cells in the notebook. I don't see a way via python to cause the current kernel to execute the rest of the cells in the notebook. I could hack something together with javascript, but that seems problematic and brittle. Thanks, Paddy -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Thu Aug 7 11:53:11 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 7 Aug 2014 08:53:11 -0700 Subject: [IPython-dev] tornado app In-Reply-To: <20140807131656.GN24532@hike.sph.harvard.edu> References: <20140807131656.GN24532@hike.sph.harvard.edu> Message-ID: We do not plan on allowed others to add new handlers to *our* app. The recommendation is that you write your own top level app that 1) defines any custom handlers you need and 2) imports any existing IPython handlers that you need. The reason for this is that the second we allow custom handlers in our app, IPython is on a slippery slope to become a full blown web app development framework. On Thu, Aug 7, 2014 at 6:16 AM, Keith wrote: > Hi, > > I have an existing tornado application and am thinking > about leveraging (moving it into) ipython notebook. > Currently the app defines its own request handlers and > websocket for communication with the client browser. > > I was thinking that the app could generate a notebook > dynamically which would be deployed on the client's > browser. Then communication to the server could take > place within the notebook's cells. Assuming this > is possible (I'm new to ipython), are there hooks in > ipython notebook that would allow me to extend tornado > i.e. add my own routes and request handlers etc? > > thanks, > Keith > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From ellisonbg at gmail.com Thu Aug 7 12:02:04 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 7 Aug 2014 09:02:04 -0700 Subject: [IPython-dev] Automatically saving notebooks as scripts has been removed In-Reply-To: References: Message-ID: I agree with @minrk that the old `--script` is not something we want to bring back and that we should figure out a good way of having that particular conversion done in the notebook UI. On Wed, Aug 6, 2014 at 2:08 PM, MinRK wrote: > The main reason it is removed is that nbformat had a notion of ?save as > .py?, which is removed in IPython 3.0 because this functionality has moved > to nbconvert in IPython 1.0. We could, of course, let the appropriate > nbconvert --to python call be implied by --script instead of the removed > Python API, but I don?t think --script was a good idea in the first place. > > So it?s not a question of whether to use nbconvert or not - you have to use > nbconvert to get .py files from .ipynb. It?s just a question of whether you > can automatically use nbconvert on save via --script. I think if we want to > support this kind of thing, more generic save/rename hooks are better than a > single special-case flag. > > -MinRK > > > > On Mon, Aug 4, 2014 at 2:39 PM, Filipe Pires Alvarenga Fernandes > wrote: >> >> Got it. Thanks for looking into this. >> >> >> >> On Mon, Aug 4, 2014 at 6:18 PM, Thomas Kluyver wrote: >>> >>> I think we may have been a bit too hasty in getting rid of that. We've >>> been talking about it for a while amongst ourselves, because it doesn't fit >>> terribly neatly into our infrastructure, but we didn't formally deprecate >>> it, and I don't think the alternative approaches are quite as convenient >>> yet. >>> >>> I've put this in the agenda for the dev meeting on Thursday. >>> >>> Thanks, >>> Thomas >>> >>> >>> On 4 August 2014 12:55, Filipe Pires Alvarenga Fernandes >>> wrote: >>>> >>>> Hello IPython devs, >>>> >>>> I was trying the latest github checkout of the IPython source and got >>>> the following message: >>>> >>>> Automatically saving notebooks as scripts has been removed. >>>> Use `ipython nbconvert --to python [notebook]` instead. >>>> >>>> I am wondering why this had to be removed. Security? >>>> >>>> I find the automatic script saving very helpful. My work-group usually >>>> commits the `.ipynb` and the `.py` side-by-side to obtain a better diff. I >>>> know that we can workaround this with nbconvert, but doing that manually is >>>> error prone, and having a script to do that will add another layer of >>>> complexity that we did not have with the automatic conversion. >>>> >>>> Well, if the automatic conversion had to go I understand, but if there >>>> is no reason can you guys consider leaving it there? >>>> >>>> Thanks, >>>> >>>> -Filipe >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From andy at payne.org Thu Aug 7 12:33:25 2014 From: andy at payne.org (Andrew Payne) Date: Thu, 7 Aug 2014 12:33:25 -0400 Subject: [IPython-dev] API to execute cells of notebook In-Reply-To: References: Message-ID: > I don't see a way via python to cause the current kernel to execute the > rest of the cells in the notebook. I could hack something together with > javascript, but that seems problematic and brittle. > There are a number of existing Javascript methods for executing cells, including execute_cell_range(start, end) and execute_all_cells(). See: https://github.com/ipython/ipython/blob/6ecbe721af987d743b7bfcec0448e4edace5f812/IPython/html/static/notebook/js/notebook.js#L1685 And this notebook includes a simple example of adding keyboard shortcut using this method. See: http://nbviewer.ipython.org/github/minrk/pydata-sv-14/blob/master/What's%20New.ipynb It's easy to generalize that to a Javascript button. -andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From paddy at paddymullen.com Thu Aug 7 12:58:01 2014 From: paddy at paddymullen.com (Paddy Mullen) Date: Thu, 7 Aug 2014 12:58:01 -0400 Subject: [IPython-dev] API to execute cells of notebook In-Reply-To: References: Message-ID: Thanks, I could do that, but it seems a bit out of place in the IPython widget framework. Imagine if cells had a hidden attribute that could be set in python. Then you could use the widget framework to implement a slideshow, or select which type of report to show. Being able to only access those apis through javascript raises the bar for use. On Thu, Aug 7, 2014 at 12:33 PM, Andrew Payne wrote: > > I don't see a way via python to cause the current kernel to execute the >> rest of the cells in the notebook. I could hack something together with >> javascript, but that seems problematic and brittle. >> > > There are a number of existing Javascript methods for executing cells, > including execute_cell_range(start, end) and execute_all_cells(). > > See: > https://github.com/ipython/ipython/blob/6ecbe721af987d743b7bfcec0448e4edace5f812/IPython/html/static/notebook/js/notebook.js#L1685 > > And this notebook includes a simple example of adding keyboard shortcut > using this method. See: > http://nbviewer.ipython.org/github/minrk/pydata-sv-14/blob/master/What's%20New.ipynb > > It's easy to generalize that to a Javascript button. > > -andy > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.yee at gmail.com Thu Aug 7 13:40:14 2014 From: raymond.yee at gmail.com (Raymond Yee) Date: Thu, 07 Aug 2014 10:40:14 -0700 Subject: [IPython-dev] IPython 2.2 release, including security fix In-Reply-To: References: Message-ID: <53E3B9FE.9090400@gmail.com> Thanks for the security fixes of 2.2. I'm using conda to install IPython. Anyone know when continuum might release IPython 2.2 via conda? Should I be contacting continuum to ask? -Raymond On 8/6/14 4:25 PM, Thomas Kluyver wrote: > Hi all, > > We've just released IPython 2.2, which rolls up a few bugfixes that > have been backported since the last release. This includes a fix for a > security issue posted a few days ago, which could have allowed an > attacker who could intercept network traffic (e.g. on insecure wifi) > to intercept a request for Mathjax and send back malicious javascript > which would run in the notebook. The fix was simply to always load > mathjax over HTTPS: > https://github.com/ipython/ipython/pull/6249 > > You can see the full list of issues and PRs for this release here: > http://ipython.org/ipython-doc/stable/whatsnew/github-stats-2.0.html#issues-closed-in-2-2-0 > > You can install this release in any of the usual ways. > http://ipython.org/install.html > > Thomas > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy at payne.org Thu Aug 7 13:50:03 2014 From: andy at payne.org (Andrew Payne) Date: Thu, 7 Aug 2014 13:50:03 -0400 Subject: [IPython-dev] API to execute cells of notebook In-Reply-To: References: Message-ID: > > I could do that, but it seems a bit out of place in the IPython widget > framework. > > Imagine if cells had a hidden attribute that could be set in python. Then > you could use the widget framework to implement a slideshow, or select > which type of report to show. Being able to only access those apis through > javascript raises the bar for use. > The challenge (I suspect) is the execution of a Web notebook is driven from Javascript. For example, the execute_all_cells() JS function I referred to earlier eventually calls notebook.execute_cell(), which invokes the execute() method on the cell. For code cells, that generates an execute message to the kernel, see: https://github.com/ipython/ipython/blob/master/IPython/html/static/notebook/js/codecell.js#L286 And the message: http://ipython.org/ipython-doc/dev/development/messaging.html#execute Note that the kernel execute message has the actual code fragment to run, not a reference to a cell. The current state of the notebook (e.g. all the code) is in the browser not the kernel (e.g. you can edit 3 code cells and then do "run all" and get the result you expect). This means a "run all" operation has to involve the browser notebook somehow. I'm not defending this design or saying what you're looking for is unreasonable. I'm just noting that there may be an impedance mismatch with way the kernel and notebooks are built (which likely derives from the team's stated design goal to keep them independent). And given that widgets are essentially a combination of kernel-side Python and browser-side Javascript with a comm backbone to pass state updates and messages, you could easily (I think) implement a widget version of what you're looking for by taking the button widget and calling the execute_all_cells() method on invocation. I hope this is helpful, -andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Aug 7 17:27:16 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 7 Aug 2014 14:27:16 -0700 Subject: [IPython-dev] IPython 2.2 release, including security fix In-Reply-To: <53E3B9FE.9090400@gmail.com> References: <53E3B9FE.9090400@gmail.com> Message-ID: On Thu, Aug 7, 2014 at 10:40 AM, Raymond Yee wrote: > Thanks for the security fixes of 2.2. I'm using conda to install > IPython. Anyone know when continuum might release IPython 2.2 via conda? > Should I be contacting continuum to ask? Some of their devs hang out here, but it's probably best to ask directly on their mailing list. We have zero control over their release schedule. Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From damianavila at gmail.com Thu Aug 7 17:30:50 2014 From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=) Date: Thu, 7 Aug 2014 18:30:50 -0300 Subject: [IPython-dev] IPython 2.2 release, including security fix In-Reply-To: References: <53E3B9FE.9090400@gmail.com> Message-ID: It will be available soon ;-) El ago 7, 2014 6:27 PM, "Fernando Perez" escribi?: > > On Thu, Aug 7, 2014 at 10:40 AM, Raymond Yee > wrote: > >> Thanks for the security fixes of 2.2. I'm using conda to install >> IPython. Anyone know when continuum might release IPython 2.2 via conda? >> Should I be contacting continuum to ask? > > > Some of their devs hang out here, but it's probably best to ask directly > on their mailing list. We have zero control over their release schedule. > > Cheers > > f > > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Aug 7 17:42:13 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 7 Aug 2014 14:42:13 -0700 Subject: [IPython-dev] tornado app In-Reply-To: References: <20140807131656.GN24532@hike.sph.harvard.edu> Message-ID: On Thu, Aug 7, 2014 at 8:53 AM, Brian Granger wrote: > The reason for this is that the second > we allow custom handlers in our app, IPython is on a slippery slope to > become a full blown web app development framework. > But, but... Our scope is well limited to *everything*! ;) -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Aug 7 17:43:59 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 7 Aug 2014 14:43:59 -0700 Subject: [IPython-dev] tornado app In-Reply-To: References: <20140807131656.GN24532@hike.sph.harvard.edu> Message-ID: Jokes aside, you may want to study a little bit how our new colaboratory works: https://github.com/jupyter/colaboratory/blob/master/colaboratory/colaboratory.py That will give you an idea of how a custom web application is built using the IPython machinery, but one that is NOT the default notebook server. Cheers, f On Thu, Aug 7, 2014 at 2:42 PM, Fernando Perez wrote: > > On Thu, Aug 7, 2014 at 8:53 AM, Brian Granger wrote: > >> The reason for this is that the second >> we allow custom handlers in our app, IPython is on a slippery slope to >> become a full blown web app development framework. >> > > But, but... Our scope is well limited to *everything*! > > ;) > > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Thu Aug 7 17:52:06 2014 From: asmeurer at gmail.com (Aaron Meurer) Date: Thu, 7 Aug 2014 16:52:06 -0500 Subject: [IPython-dev] IPython 2.2 release, including security fix In-Reply-To: <53E3B9FE.9090400@gmail.com> References: <53E3B9FE.9090400@gmail.com> Message-ID: Some time next week at the latest. Aaron Meurer On Thu, Aug 7, 2014 at 12:40 PM, Raymond Yee wrote: > Thanks for the security fixes of 2.2. I'm using conda to install > IPython. Anyone know when continuum might release IPython 2.2 via conda? > Should I be contacting continuum to ask? > > -Raymond > > > On 8/6/14 4:25 PM, Thomas Kluyver wrote: > > Hi all, > > We've just released IPython 2.2, which rolls up a few bugfixes that have > been backported since the last release. This includes a fix for a security > issue posted a few days ago, which could have allowed an attacker who could > intercept network traffic (e.g. on insecure wifi) to intercept a request > for Mathjax and send back malicious javascript which would run in the > notebook. The fix was simply to always load mathjax over HTTPS: > https://github.com/ipython/ipython/pull/6249 > > You can see the full list of issues and PRs for this release here: > > http://ipython.org/ipython-doc/stable/whatsnew/github-stats-2.0.html#issues-closed-in-2-2-0 > > You can install this release in any of the usual ways. > http://ipython.org/install.html > > Thomas > > > _______________________________________________ > IPython-dev mailing listIPython-dev at scipy.orghttp://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From illanos at cal.berkeley.edu Thu Aug 7 23:22:25 2014 From: illanos at cal.berkeley.edu (Imelda Llanos de Luna) Date: Thu, 7 Aug 2014 20:22:25 -0700 Subject: [IPython-dev] requesting feedback: IPython-BusinessNeed draft report Message-ID: Hi,I'm a Business Analyst, volunteering my services. I am performing analysis of the IPython project, and I have completed a DRAFT of the Business Need section of the report. For the draft, I performed a document analysis to help write the draft, pulling written sections that existed on other documents. Please review the draft, and provide feedback by Thurs (8/14). I welcome it, and any other corrections and clarifications that can be provided. Again, this is only one section of the report, solely focused on defining the business need at a high level. There are more sections left to write: http://nbviewer.ipython.org/github/ofcloudpeople/Business-Analysis/blob/IPythonProject/IPython-BusinessNeed.ipynb . Note also: I had to close the gap between the time the project started in 2001 to the present. I believe I was able to close it as written. If yes , then any writing in the next sections can focus on present goals and objectives, requirements, etc. Thank you. Sincerely,Imelda -------------- next part -------------- An HTML attachment was scrubbed... URL: From zvoros at gmail.com Fri Aug 8 00:46:17 2014 From: zvoros at gmail.com (=?UTF-8?B?Wm9sdMOhbiBWw7Zyw7Zz?=) Date: Fri, 08 Aug 2014 06:46:17 +0200 Subject: [IPython-dev] mathjax equation number is incremented on cntr-enter Message-ID: <53E45619.4040302@gmail.com> Hi all, I don't know, whether this is a known issue (a quick search did not turn up anything), in fact, I don't even know, whether this is an issue at all, but I believe, the markdown cell did behave differently in the past. In short, it seems to me that mathjax increments the equation number, whenever I press Cntr-Enter on a markdown cell. Here is my notebook, and here is the code that produces the problem: \begin{equation} x = y \end{equation} I haven't removed anything, yet, the equation is labelled (4). Can someone comment on this? In particular, whether this was intended? Is this, perhaps, related to the security problem that was floating around recently? I am running the latest code from master. Cheers, Zolt?n -------------- next part -------------- A non-text attachment was scrubbed... Name: Screenshot from 2014-08-08 06:37:04.png Type: image/png Size: 15159 bytes Desc: not available URL: From fperez.net at gmail.com Fri Aug 8 02:29:21 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 7 Aug 2014 23:29:21 -0700 Subject: [IPython-dev] requesting feedback: IPython-BusinessNeed draft report In-Reply-To: References: Message-ID: Hi all, I should have done this prior, but better late than never.. :) I just wanted to introduce Imelda, who contacted us a few days ago and volu I hope this will nteered her time and skills to help out IPython. Yesterday met at Berkeley with the local part of the team as well as Katy Huff from Scipy. Imelda volunteered to help the project with writing from a different perspective than what we typically write: not technical docs but business/project analysis. That's a skill set different from our typical one, so I'm happy to see where this experiment can take us. We decided a good starting point could be for her to take the view of a business analyst and look at what the project looks like *from the outside*, without much detailed guidance from us at all. I'm sure that will be a sobering diagnostic of some of our limitations in terms of docs, project presentation, website, etc. Cheers f On Thu, Aug 7, 2014 at 8:22 PM, Imelda Llanos de Luna < illanos at cal.berkeley.edu> wrote: > Hi, > I'm a Business Analyst, volunteering my services. I am performing > analysis of the IPython project, and I have completed a DRAFT of the > Business Need section of the report. For the draft, I performed a document > analysis to help write the draft, pulling written sections that existed on > other documents. Please review the draft, and provide feedback by Thurs > (8/14). I welcome it, and any other corrections and clarifications that > can be provided. > > Again, this is only one section of the report, solely focused on defining > the business need at a high level. There are more sections left to write: > http://nbviewer.ipython.org/github/ofcloudpeople/Business-Analysis/blob/IPythonProject/IPython-BusinessNeed.ipynb . > Note also: I had to close the gap between the time the project started > in 2001 to the present. I believe I was able to close it as written. If > yes , then any writing in the next sections can focus on present goals and > objectives, requirements, etc. > > Thank you. > > Sincerely, > Imelda > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From markbak at gmail.com Fri Aug 8 09:27:39 2014 From: markbak at gmail.com (Mark Bakker) Date: Fri, 8 Aug 2014 15:27:39 +0200 Subject: [IPython-dev] changing working directory from IPython server Message-ID: Hello List, I know this has been discussed before, but it doesn't seem to be resolved. Currently (v.2.1 on my machine) I can start an IPython notebook server anywhere. Then click and open an IPython Notebook anywhere else. That IPython Notebook then gets copied to the directory where I started the server. Problem with that workflow is that that is rarely what you want it to do. Is there any way, or are there any plans, to be able to change the working directory from the Notebook server page (the one that opens when you type 'ipython notebook')? It would make my life, and that of my students, SO much easier. Thanks for all the great work. Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Fri Aug 8 09:48:59 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 8 Aug 2014 06:48:59 -0700 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: <6A43CDB9-EB1A-4F21-87FF-A3136AC47749@gmail.com> The notebook shouldn't behave as you describe in 2.1. Notebooks should be saved in the directory they are created. There is something fishy going on... Sent from my iPhone > On Aug 8, 2014, at 6:27 AM, Mark Bakker wrote: > > Hello List, > > I know this has been discussed before, but it doesn't seem to be resolved. > > Currently (v.2.1 on my machine) I can start an IPython notebook server anywhere. > Then click and open an IPython Notebook anywhere else. > That IPython Notebook then gets copied to the directory where I started the server. > > Problem with that workflow is that that is rarely what you want it to do. > > Is there any way, or are there any plans, to be able to change the working directory from the Notebook server page (the one that opens when you type 'ipython notebook')? It would make my life, and that of my students, SO much easier. > > Thanks for all the great work. > > Mark > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From markbak at gmail.com Fri Aug 8 10:00:31 2014 From: markbak at gmail.com (Mark Bakker) Date: Fri, 8 Aug 2014 16:00:31 +0200 Subject: [IPython-dev] changing working directory from IPython server Message-ID: Brian, I sure hope I am doing something wrong. What I do is click on 'To import a notebook, drag the file onto the listing below or *click here*. When I click there, a window pops up, I got to the directory with my ipython file, click on it, and then it shows up in the list of files, with next to it a blue Upload button. When I click on the Upload button the notebook is copied to the directory where I started the IPython Notebook. But I don't want to copy the Notebook to that directory (also cause the data files associated with the notebook are still in the old directory). Am I doing something wrong? Mark Brian Granger wrote: The notebook shouldn't behave as you describe in 2.1. Notebooks should be saved in the directory they are created. There is something fishy going on... Sent from my iPhone On Fri, Aug 8, 2014 at 3:27 PM, Mark Bakker wrote: > Hello List, > > I know this has been discussed before, but it doesn't seem to be resolved. > > Currently (v.2.1 on my machine) I can start an IPython notebook server > anywhere. > Then click and open an IPython Notebook anywhere else. > That IPython Notebook then gets copied to the directory where I started > the server. > > Problem with that workflow is that that is rarely what you want it to do. > > Is there any way, or are there any plans, to be able to change the working > directory from the Notebook server page (the one that opens when you type > 'ipython notebook')? It would make my life, and that of my students, SO > much easier. > > Thanks for all the great work. > > Mark > -------------- next part -------------- An HTML attachment was scrubbed... URL: From markbak at gmail.com Fri Aug 8 10:03:21 2014 From: markbak at gmail.com (Mark Bakker) Date: Fri, 8 Aug 2014 16:03:21 +0200 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: Oops, I am running IPython 2.0. Is this better in 2.1? If so, then very sorry for the noise and have a nice weekend. (If not: help!) Mark On Fri, Aug 8, 2014 at 4:00 PM, Mark Bakker wrote: > Brian, I sure hope I am doing something wrong. > What I do is click on 'To import a notebook, drag the file onto the > listing below or *click here*. When I click there, a window pops up, I got > to the directory with my ipython file, click on it, and then it shows up in > the list of files, with next to it a blue Upload button. When I click on > the Upload button the notebook is copied to the directory where I started > the IPython Notebook. > > But I don't want to copy the Notebook to that directory (also cause the > data files associated with the notebook are still in the old directory). Am > I doing something wrong? > > Mark > > Brian Granger wrote: > > The notebook shouldn't behave as you describe in 2.1. Notebooks should be > saved in the directory they are created. There is something fishy going > on... > > Sent from my iPhone > > > On Fri, Aug 8, 2014 at 3:27 PM, Mark Bakker wrote: > >> Hello List, >> >> I know this has been discussed before, but it doesn't seem to be resolved. >> >> Currently (v.2.1 on my machine) I can start an IPython notebook server >> anywhere. >> Then click and open an IPython Notebook anywhere else. >> That IPython Notebook then gets copied to the directory where I started >> the server. >> >> Problem with that workflow is that that is rarely what you want it to do. >> >> Is there any way, or are there any plans, to be able to change the >> working directory from the Notebook server page (the one that opens when >> you type 'ipython notebook')? It would make my life, and that of my >> students, SO much easier. >> >> Thanks for all the great work. >> >> Mark >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Fri Aug 8 10:39:30 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 8 Aug 2014 07:39:30 -0700 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: <9127FEAE-5736-4EE1-A43F-2CCD3549DF64@gmail.com> Ahh sorry, I missed you are uploading a notebook. That may be a bug, but I don't remember. Can you open a github issue for thus and we will test ourselves? Sent from my iPhone > On Aug 8, 2014, at 7:03 AM, Mark Bakker wrote: > > Oops, I am running IPython 2.0. > Is this better in 2.1? > If so, then very sorry for the noise and have a nice weekend. > (If not: help!) > Mark > > >> On Fri, Aug 8, 2014 at 4:00 PM, Mark Bakker wrote: >> Brian, I sure hope I am doing something wrong. >> What I do is click on 'To import a notebook, drag the file onto the listing below or *click here*. When I click there, a window pops up, I got to the directory with my ipython file, click on it, and then it shows up in the list of files, with next to it a blue Upload button. When I click on the Upload button the notebook is copied to the directory where I started the IPython Notebook. >> >> But I don't want to copy the Notebook to that directory (also cause the data files associated with the notebook are still in the old directory). Am I doing something wrong? >> >> Mark >> >> Brian Granger wrote: >> >> The notebook shouldn't behave as you describe in 2.1. Notebooks should be saved in the directory they are created. There is something fishy going on... >> >> Sent from my iPhone >> >> >>> On Fri, Aug 8, 2014 at 3:27 PM, Mark Bakker wrote: >>> Hello List, >>> >>> I know this has been discussed before, but it doesn't seem to be resolved. >>> >>> Currently (v.2.1 on my machine) I can start an IPython notebook server anywhere. >>> Then click and open an IPython Notebook anywhere else. >>> That IPython Notebook then gets copied to the directory where I started the server. >>> >>> Problem with that workflow is that that is rarely what you want it to do. >>> >>> Is there any way, or are there any plans, to be able to change the working directory from the Notebook server page (the one that opens when you type 'ipython notebook')? It would make my life, and that of my students, SO much easier. >>> >>> Thanks for all the great work. >>> >>> Mark > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Fri Aug 8 10:47:45 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 8 Aug 2014 07:47:45 -0700 Subject: [IPython-dev] requesting feedback: IPython-BusinessNeed draft report In-Reply-To: References: Message-ID: <7F928AF2-EF31-4A3B-910A-F54BE90CD91F@gmail.com> Great, I will have a look! Sent from my iPhone > On Aug 7, 2014, at 11:29 PM, Fernando Perez wrote: > > Hi all, > > I should have done this prior, but better late than never.. :) > > I just wanted to introduce Imelda, who contacted us a few days ago and volu > > I hope this will nteered her time and skills to help out IPython. Yesterday met at Berkeley with the local part of the team as well as Katy Huff from Scipy. > > Imelda volunteered to help the project with writing from a different perspective than what we typically write: not technical docs but business/project analysis. That's a skill set different from our typical one, so I'm happy to see where this experiment can take us. > > We decided a good starting point could be for her to take the view of a business analyst and look at what the project looks like *from the outside*, without much detailed guidance from us at all. I'm sure that will be a sobering diagnostic of some of our limitations in terms of docs, project presentation, website, etc. > > Cheers > > f > > >> On Thu, Aug 7, 2014 at 8:22 PM, Imelda Llanos de Luna wrote: >> Hi, >> I'm a Business Analyst, volunteering my services. I am performing analysis of the IPython project, and I have completed a DRAFT of the Business Need section of the report. For the draft, I performed a document analysis to help write the draft, pulling written sections that existed on other documents. Please review the draft, and provide feedback by Thurs (8/14). I welcome it, and any other corrections and clarifications that can be provided. >> >> Again, this is only one section of the report, solely focused on defining the business need at a high level. There are more sections left to write: http://nbviewer.ipython.org/github/ofcloudpeople/Business-Analysis/blob/IPythonProject/IPython-BusinessNeed.ipynb . Note also: I had to close the gap between the time the project started in 2001 to the present. I believe I was able to close it as written. If yes , then any writing in the next sections can focus on present goals and objectives, requirements, etc. >> >> Thank you. >> >> Sincerely, >> Imelda >> >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Fri Aug 8 13:51:41 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 8 Aug 2014 10:51:41 -0700 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: On 8 August 2014 07:00, Mark Bakker wrote: > What I do is click on 'To import a notebook, drag the file onto the > listing below or *click here*. When I click there, a window pops up, I got > to the directory with my ipython file, click on it, and then it shows up in > the list of files, with next to it a blue Upload button. When I click on > the Upload button the notebook is copied to the directory where I started > the IPython Notebook. > > But I don't want to copy the Notebook to that directory (also cause the > data files associated with the notebook are still in the old directory). Am > I doing something wrong? > That is by design, though I agree that it's rather strange. An unfortunate result of building the notebook as a browser UI and a web server is that some parts behave like a desktop app, and some behave like a web app. The original notion of files was designed like a web app - you upload files to the server, and it works with the uploaded copy, ignoring the file it originally came from (if it wasn't running locally, it wouldn't be able to access the original file). So don't upload notebooks. Start the notebook in the directory where your notebooks are, or in a parent directory of that. My nbopen tool helps with this - it opens a notebook in an already running server, or starts a new one if necessary. https://github.com/takluyver/nbopen Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.m.bray at gmail.com Fri Aug 8 17:55:14 2014 From: erik.m.bray at gmail.com (Erik Bray) Date: Fri, 8 Aug 2014 17:55:14 -0400 Subject: [IPython-dev] Automatically saving notebooks as scripts has been removed In-Reply-To: References: Message-ID: On Thu, Aug 7, 2014 at 12:02 PM, Brian Granger wrote: > I agree with @minrk that the old `--script` is not something we want > to bring back and that we should figure out a good way of having that > particular conversion done in the notebook UI. Would somebody please be willing to summarize the differences between nbconvert --to python and the aforementioned --script option? I think that the format that is output from the notebook by "Save as .py..." is very handy to have. And in fact I was just recently singing its praises as a decent way to go between code in a notebook to an importable Python module (and perhaps back again?) So it's definitely a feature that should exist in some form I think. Erik > On Wed, Aug 6, 2014 at 2:08 PM, MinRK wrote: >> The main reason it is removed is that nbformat had a notion of ?save as >> .py?, which is removed in IPython 3.0 because this functionality has moved >> to nbconvert in IPython 1.0. We could, of course, let the appropriate >> nbconvert --to python call be implied by --script instead of the removed >> Python API, but I don?t think --script was a good idea in the first place. >> >> So it?s not a question of whether to use nbconvert or not - you have to use >> nbconvert to get .py files from .ipynb. It?s just a question of whether you >> can automatically use nbconvert on save via --script. I think if we want to >> support this kind of thing, more generic save/rename hooks are better than a >> single special-case flag. >> >> -MinRK >> >> >> >> On Mon, Aug 4, 2014 at 2:39 PM, Filipe Pires Alvarenga Fernandes >> wrote: >>> >>> Got it. Thanks for looking into this. >>> >>> >>> >>> On Mon, Aug 4, 2014 at 6:18 PM, Thomas Kluyver wrote: >>>> >>>> I think we may have been a bit too hasty in getting rid of that. We've >>>> been talking about it for a while amongst ourselves, because it doesn't fit >>>> terribly neatly into our infrastructure, but we didn't formally deprecate >>>> it, and I don't think the alternative approaches are quite as convenient >>>> yet. >>>> >>>> I've put this in the agenda for the dev meeting on Thursday. >>>> >>>> Thanks, >>>> Thomas >>>> >>>> >>>> On 4 August 2014 12:55, Filipe Pires Alvarenga Fernandes >>>> wrote: >>>>> >>>>> Hello IPython devs, >>>>> >>>>> I was trying the latest github checkout of the IPython source and got >>>>> the following message: >>>>> >>>>> Automatically saving notebooks as scripts has been removed. >>>>> Use `ipython nbconvert --to python [notebook]` instead. >>>>> >>>>> I am wondering why this had to be removed. Security? >>>>> >>>>> I find the automatic script saving very helpful. My work-group usually >>>>> commits the `.ipynb` and the `.py` side-by-side to obtain a better diff. I >>>>> know that we can workaround this with nbconvert, but doing that manually is >>>>> error prone, and having a script to do that will add another layer of >>>>> complexity that we did not have with the automatic conversion. >>>>> >>>>> Well, if the automatic conversion had to go I understand, but if there >>>>> is no reason can you guys consider leaving it there? >>>>> >>>>> Thanks, >>>>> >>>>> -Filipe >>>>> >>>>> _______________________________________________ >>>>> IPython-dev mailing list >>>>> IPython-dev at scipy.org >>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>>> >>>> >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>>> >>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From takowl at gmail.com Fri Aug 8 18:03:55 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 8 Aug 2014 15:03:55 -0700 Subject: [IPython-dev] Automatically saving notebooks as scripts has been removed In-Reply-To: References: Message-ID: On 8 August 2014 14:55, Erik Bray wrote: > Would somebody please be willing to summarize the differences between > > nbconvert --to python > > and the aforementioned --script option? --script is an option to the notebook server, so that whenever you save a notebook, you get a copy of the .py script as well. nbconvert has to be run as a separate command when you want to do a conversion. Currently, --script relies on a separate mechanism to write notebooks to .py files, but that mechanism is being removed. We'll reinstate --script for IPython 3, using the nbconvert machinery to write notebooks, but it will be deprecated. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ocefpaf at gmail.com Fri Aug 8 18:25:10 2014 From: ocefpaf at gmail.com (Filipe Pires Alvarenga Fernandes) Date: Fri, 8 Aug 2014 19:25:10 -0300 Subject: [IPython-dev] Automatically saving notebooks as scripts has been removed In-Reply-To: References: Message-ID: > > We'll reinstate --script for IPython 3, using the nbconvert machinery to > write notebooks, but it will be deprecated. > So the final decision is to remove... I will try one last pitch here. I believe that, even for people using the Jupyter with other languages (Julia, Haskell, etc), the "--script" option is useful. My group uses the notebook for visualization and the "script" for diffs on github by committing them together*. Having the script updated/saved automatically with the "--script" flag is a big help for our workflow. Converting the notebooks via the command line or using the UI would be regression for our use case. I am sure that someone will come up with a %magic or a github hook for that, but this feature is already there and, IMHO, I do not see a reason for it to go *Or at least until a project like nbdiff takes off. Thanks, -Filipe -------------- next part -------------- An HTML attachment was scrubbed... URL: From jklymak at gmail.com Sun Aug 10 11:39:03 2014 From: jklymak at gmail.com (Jody Klymak) Date: Sun, 10 Aug 2014 08:39:03 -0700 Subject: [IPython-dev] Preferred way to customize CSS for something that will be displayed on nbviewer? In-Reply-To: References: Message-ID: <8149380B-9E8B-4B25-8067-934AFCF5B1E2@gmail.com> Hi all Has this been updated yet? I'd like to share ipython notebooks on nbviewer for teaching and would like to change the CSS to make the pages easier to read. What is the current best practice? Thanks. Jody On Feb 20, 2014, at 2:54 AM, Matthias Bussonnier wrote: > There should be a discussion on IPython user about that from a few weeks ago. > > We are thinking on metadata in notebook level that nbviewer could look at that allow to select a theme on nbviewer. > > IPEP and code to do so have to be written. > > Envoy? de mon iPhone > > Le 20 f?vr. 2014 ? 04:58, Aron Ahmadia a ?crit : > >> Let's say I've developed a snazzy CSS file and I want it so that an uploaded notebook being viewed on nbviewer uses my custom CSS. >> >> I've seen this bit of code in the wild[1]: >> >> from IPython.core.display import HTML >> def css_styling(): >> styles = open("../styles/custom.css", "r").read() >> return HTML(styles) >> css_styling() >> >> But this seems a bit awkward, since I'm effectively just piping the CSS file into the notebook's rendered HTML. Does nbviewer know to look anywhere for CSS, or is this the best we have right now? >> >> -A >> >> [1] http://nbviewer.ipython.org/github/barbagroup/AeroPython/blob/master/lessons/00_Lesson00_QuickPythonIntro.ipynb >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From aron at ahmadia.net Sun Aug 10 11:40:00 2014 From: aron at ahmadia.net (Aron Ahmadia) Date: Sun, 10 Aug 2014 11:40:00 -0400 Subject: [IPython-dev] Preferred way to customize CSS for something that will be displayed on nbviewer? In-Reply-To: <8149380B-9E8B-4B25-8067-934AFCF5B1E2@gmail.com> References: <8149380B-9E8B-4B25-8067-934AFCF5B1E2@gmail.com> Message-ID: I think the latest discussion has been here: https://github.com/ipython/ipython/issues/5921 On Sun, Aug 10, 2014 at 11:39 AM, Jody Klymak wrote: > Hi all > > Has this been updated yet? I'd like to share ipython notebooks on > nbviewer for teaching and would like to change the CSS to make the pages > easier to read. What is the current best practice? > > Thanks. Jody > > > On Feb 20, 2014, at 2:54 AM, Matthias Bussonnier < > bussonniermatthias at gmail.com> wrote: > > There should be a discussion on IPython user about that from a few weeks > ago. > > We are thinking on metadata in notebook level that nbviewer could look at > that allow to select a theme on nbviewer. > > IPEP and code to do so have to be written. > > Envoy? de mon iPhone > > Le 20 f?vr. 2014 ? 04:58, Aron Ahmadia a ?crit : > > Let's say I've developed a snazzy CSS file and I want it so that an > uploaded notebook being viewed on nbviewer uses my custom CSS. > > I've seen this bit of code in the wild[1]: > > from IPython.core.display import HTML > def css_styling(): > styles = open("../styles/custom.css", "r").read() > return HTML(styles) > css_styling() > > But this seems a bit awkward, since I'm effectively just piping the CSS > file into the notebook's rendered HTML. Does nbviewer know to look > anywhere for CSS, or is this the best we have right now? > > -A > > [1] > http://nbviewer.ipython.org/github/barbagroup/AeroPython/blob/master/lessons/00_Lesson00_QuickPythonIntro.ipynb > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Sun Aug 10 20:22:03 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 10 Aug 2014 17:22:03 -0700 Subject: [IPython-dev] Separating the notebook from the file manager+text editor+terminal Message-ID: We've often said that, for people accessing the notebook on a remote server, we need to provide certain basic utilities - like the dashboard, which is now under pressure to evolve into a file manager, and an in-browser terminal and text editor, both of which we intend to add to the notebook. I propose that we separate development and packaging of the notebook from the project to assemble this trio. The new project would live under the Jupyter org, and be called something like 'utility-apps'. I see several advantages of this: - A different crowd of developers may be attracted to work on that, without having to know anything about the internals of IPython's machinery for handling and executing code. For instance, I'm pushing back on turning the dashboard into a full blown file manager, because I think that adds a lot of extra complexity in a codebase that's already complex, but if it was a separate project, it could grow into a file manager without being a problem. - People running the notebook locally may want to install just the notebook component, and not poor web-based clones of applications which they have well developed equivalents for already installed. - There are lots of potential users of the file manager/text editor/terminal trio who probably wouldn't use the notebook: for instance, it could be a basic admin interface to any server. - It could become a base on which people could write other HTML applications to meet their own needs. Following our usual MO, I wouldn't try to design it like this straight away, but keep it in mind, and evolve the APIs as things mature. There are still plenty of questions to be answered. How would this new component interface with the notebook server? What would the notebook look like without the file manager installed? But these could certainly be worked out, and I think they're worth working out in order to keep the scope of the notebook project in check. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From moritz.beber at gmail.com Mon Aug 11 08:52:13 2014 From: moritz.beber at gmail.com (Moritz Beber) Date: Mon, 11 Aug 2014 14:52:13 +0200 Subject: [IPython-dev] IPython.parallel slow push Message-ID: Dear all, I often find myself in the situation where I use IPython.parallel to push a bunch of data to the kernels (around 2 GB) and then do calculations over a large parameter space on that data. This is convenient and simple enough to do but copying the data takes a large amount of time (it scales with the number of kernels). Is this something to be avoided altogether, are there ways to speed it up? I'd welcome any pointers. Best, Moritz -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Mon Aug 11 09:06:46 2014 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 11 Aug 2014 08:06:46 -0500 Subject: [IPython-dev] IPython.parallel slow push In-Reply-To: References: Message-ID: If I understand your use case correctly, each computation node needs a copy of all 2GB of the same data? For reference, how does the transfer rate compare to rsync-ing the same data over SSH? These may be helpful: * https://en.wikipedia.org/wiki/Locality_of_reference * https://en.wikipedia.org/wiki/Clustered_file_system#Examples_2 * https://en.wikipedia.org/wiki/MapReduce#Dataflow * https://spark.apache.org/ * http://continuum.io/blog/blaze : > Unlike NumPy, Blaze is designed to handle out-of-core computations on large datasets that exceed the system memory capacity, as well as on distributed and streaming data. Blaze is able to operate on datasets transparently as if they behaved like in-memory NumPy arrays. > > We aim to allow analysts and scientists to productively write robust and efficient code, without getting bogged down in the details of how to distribute computation, or worse, how to transport and convert data between databases, formats, proprietary data warehouses, and other silos. -- Wes Turner On Mon, Aug 11, 2014 at 7:52 AM, Moritz Beber wrote: > Dear all, > > I often find myself in the situation where I use IPython.parallel to push a > bunch of data to the kernels (around 2 GB) and then do calculations over a > large parameter space on that data. This is convenient and simple enough to > do but copying the data takes a large amount of time (it scales with the > number of kernels). > > Is this something to be avoided altogether, are there ways to speed it up? > > I'd welcome any pointers. > > Best, > Moritz > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From moritz.beber at gmail.com Mon Aug 11 09:35:32 2014 From: moritz.beber at gmail.com (Moritz Beber) Date: Mon, 11 Aug 2014 15:35:32 +0200 Subject: [IPython-dev] IPython.parallel slow push In-Reply-To: References: Message-ID: Hi Wes, Thank you for the quick response. On Mon, Aug 11, 2014 at 3:06 PM, Wes Turner wrote: > If I understand your use case correctly, each computation node needs a > copy of all 2GB of the same data? > > For reference, how does the transfer rate compare to rsync-ing the > same data over SSH? > I should have been more clear, it's actually a local cluster, i.e., same host just multiple local kernels. So I have one process that loads the data, does some pre-processing and then creates a dictionary that it pushes to the namespace of the other kernels. I understand that the data needs to go through ZMQ but since it's on the same machine I expected it to be faster (with 64 kernels it takes about 15 min). If desired I can come up with a minimal notebook. Cheers, Moritz -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Mon Aug 11 09:56:16 2014 From: wes.turner at gmail.com (Wes Turner) Date: Mon, 11 Aug 2014 08:56:16 -0500 Subject: [IPython-dev] IPython.parallel slow push In-Reply-To: References: Message-ID: On Mon, Aug 11, 2014 at 8:35 AM, Moritz Beber wrote: > Hi Wes, > > Thank you for the quick response. > > > On Mon, Aug 11, 2014 at 3:06 PM, Wes Turner wrote: >> >> If I understand your use case correctly, each computation node needs a >> copy of all 2GB of the same data? >> >> For reference, how does the transfer rate compare to rsync-ing the >> same data over SSH? > > > I should have been more clear, it's actually a local cluster, i.e., same > host just multiple local kernels. Got it. So the network is probably not the primary bottleneck. > So I have one process that loads the data, > does some pre-processing and then creates a dictionary that it pushes to the > namespace of the other kernels. A 2GB dictionary from RAM to { ... }. With Celery, similar workflows can be modeled with a "Chord". [1] > I understand that the data needs to go > through ZMQ but since it's on the same machine I expected it to be faster > (with 64 kernels it takes about 15 min). TBH, I'm not too familiar with IPython.parallel. This [2] seems to suggest that anything that isn't a buffer, str/bytes, or numpy array is pickled and copied. Would it be faster to ETL into something like HDF5 (e.g. w/ Pandas/PyTables) and just synchronize the dataset URI? [1] http://celery.readthedocs.org/en/latest/userguide/canvas.html#chords [2] http://ipython.org/ipython-doc/dev/parallel/parallel_details.html#what-is-sendable > If desired I can come up with a > minimal notebook. > > Cheers, > Moritz > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Wes Turner From ellisonbg at gmail.com Mon Aug 11 10:51:22 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 11 Aug 2014 07:51:22 -0700 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: I don't remember that we made the decision to have this behavior. It is possibly I just don't remember, but today I would consider that to be a bug. Mark, could you open a GitHub issue on this to trigger a discussion. Cheers, Brian On Fri, Aug 8, 2014 at 10:51 AM, Thomas Kluyver wrote: > On 8 August 2014 07:00, Mark Bakker wrote: >> >> What I do is click on 'To import a notebook, drag the file onto the >> listing below or *click here*. When I click there, a window pops up, I got >> to the directory with my ipython file, click on it, and then it shows up in >> the list of files, with next to it a blue Upload button. When I click on the >> Upload button the notebook is copied to the directory where I started the >> IPython Notebook. >> >> But I don't want to copy the Notebook to that directory (also cause the >> data files associated with the notebook are still in the old directory). Am >> I doing something wrong? > > > That is by design, though I agree that it's rather strange. An unfortunate > result of building the notebook as a browser UI and a web server is that > some parts behave like a desktop app, and some behave like a web app. The > original notion of files was designed like a web app - you upload files to > the server, and it works with the uploaded copy, ignoring the file it > originally came from (if it wasn't running locally, it wouldn't be able to > access the original file). > > So don't upload notebooks. Start the notebook in the directory where your > notebooks are, or in a parent directory of that. > > My nbopen tool helps with this - it opens a notebook in an already running > server, or starts a new one if necessary. > https://github.com/takluyver/nbopen > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From takowl at gmail.com Mon Aug 11 13:03:56 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 11 Aug 2014 10:03:56 -0700 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: On 11 August 2014 07:51, Brian Granger wrote: > I don't remember that we made the decision to have this behavior. It > is possibly I just don't remember, but today I would consider that to > be a bug. Mark, could you open a GitHub issue on this to trigger a > discussion. > I'm pretty sure that what Mark describes is how we expect it to work - when you upload a file, the server doesn't know where in your filesystem that data is coming from, so it can't switch to that directory and use the original notebook there, which I think is what Mark wants. The disconnect is that the UI implicitly encourages uploading, although that doesn't really make sense when you're running the server as a local application. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Mon Aug 11 13:07:54 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 11 Aug 2014 10:07:54 -0700 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: Oh, I might have understood. Mark can you clarify what exactly you are seeing and what you expect to see? On Mon, Aug 11, 2014 at 10:03 AM, Thomas Kluyver wrote: > On 11 August 2014 07:51, Brian Granger wrote: >> >> I don't remember that we made the decision to have this behavior. It >> is possibly I just don't remember, but today I would consider that to >> be a bug. Mark, could you open a GitHub issue on this to trigger a >> discussion. > > > I'm pretty sure that what Mark describes is how we expect it to work - when > you upload a file, the server doesn't know where in your filesystem that > data is coming from, so it can't switch to that directory and use the > original notebook there, which I think is what Mark wants. The disconnect is > that the UI implicitly encourages uploading, although that doesn't really > make sense when you're running the server as a local application. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From markbak at gmail.com Mon Aug 11 16:11:15 2014 From: markbak at gmail.com (Mark Bakker) Date: Mon, 11 Aug 2014 22:11:15 +0200 Subject: [IPython-dev] changing working directory from IPython server Message-ID: Thanks for considering this issue. The behavior now: I type: ipython notebook to start a notebook. At the dashboard, I click on 'To import a notebook, drag the file onto the listing below or *click here*. When I click there, a window pops up, I go to the directory with my notebook file, click on it, and then it shows up in the list of files, with next to it a blue Upload button. When I click on the Upload button the notebook is copied to the directory where I started the IPython Notebook. The problem: The notebook gets copied to the directory where I started ipython notebook. That is not very useful, as all associated data files are still in the directory where the notebook originated. Does that make it clear? Should I file a bug report? Mark ps. It seems that the Anaconda guys are also expecting the behavior of the Notebook to change. They now have a useful 'start notebook' icon, but that starts in 'Documents/IPython Notebooks. And that faces the same problem as I described above. From: Brian Granger > Subject: Re: [IPython-dev] changing working directory from IPython > server > To: IPython developers list > Message-ID: > < > CAH4pYpTsm2Vykb27Zaa4kJibmvaN14nSgAg6uq0NBASoQCSXgA at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > Oh, I might have understood. > > Mark can you clarify what exactly you are seeing and what you expect to > see? > > On Mon, Aug 11, 2014 at 10:03 AM, Thomas Kluyver wrote: > > On 11 August 2014 07:51, Brian Granger wrote: > >> > >> I don't remember that we made the decision to have this behavior. It > >> is possibly I just don't remember, but today I would consider that to > >> be a bug. Mark, could you open a GitHub issue on this to trigger a > >> discussion. > > > > > > I'm pretty sure that what Mark describes is how we expect it to work - > when > > you upload a file, the server doesn't know where in your filesystem that > > data is coming from, so it can't switch to that directory and use the > > original notebook there, which I think is what Mark wants. The > disconnect is > > that the UI implicitly encourages uploading, although that doesn't really > > make sense when you're running the server as a local application. > > > > Thomas > > > - > Brian E. Granger > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From markbak at gmail.com Mon Aug 11 16:14:01 2014 From: markbak at gmail.com (Mark Bakker) Date: Mon, 11 Aug 2014 22:14:01 +0200 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: I forgot to add the expected behavior: I expect that once I select the desired notebook in the dashboard, that it is opened in the directory where it originates. So it doesn't get copied and all the associated files are there in the same directory. On Mon, Aug 11, 2014 at 10:11 PM, Mark Bakker wrote: > Thanks for considering this issue. > > The behavior now: > I type: ipython notebook to start a notebook. > At the dashboard, I click on 'To import a notebook, drag the file onto > the listing below or *click here*. When I click there, a window pops up, I > go to the directory with my notebook file, click on it, and then it shows > up in the list of files, with next to it a blue Upload button. When I click > on the Upload button the notebook is copied to the directory where I > started the IPython Notebook. > > The problem: The notebook gets copied to the directory where I started > ipython notebook. That is not very useful, as all associated data files are > still in the directory where the notebook originated. > > Does that make it clear? Should I file a bug report? > > Mark > > ps. It seems that the Anaconda guys are also expecting the behavior of the > Notebook to change. They now have a useful 'start notebook' icon, but that > starts in 'Documents/IPython Notebooks. And that faces the same problem as > I described above. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Mon Aug 11 16:20:47 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 11 Aug 2014 13:20:47 -0700 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: Mark, OK I think I understand, and I don't consider this a bug. If the notebook you are starting out with is on the same machine, there is no reason to "upload" that notebook. Just navigate the notebook dashboard to the directory where that notebook already is and open it. Is there some reason you can't do that and are instead trying to upload the notebook. Cheers, Brian On Mon, Aug 11, 2014 at 1:11 PM, Mark Bakker wrote: > Thanks for considering this issue. > > The behavior now: > I type: ipython notebook to start a notebook. > At the dashboard, I click on 'To import a notebook, drag the file onto the > listing below or *click here*. When I click there, a window pops up, I go to > the directory with my notebook file, click on it, and then it shows up in > the list of files, with next to it a blue Upload button. When I click on the > Upload button the notebook is copied to the directory where I started the > IPython Notebook. > > The problem: The notebook gets copied to the directory where I started > ipython notebook. That is not very useful, as all associated data files are > still in the directory where the notebook originated. > > Does that make it clear? Should I file a bug report? > > Mark > > ps. It seems that the Anaconda guys are also expecting the behavior of the > Notebook to change. They now have a useful 'start notebook' icon, but that > starts in 'Documents/IPython Notebooks. And that faces the same problem as I > described above. > > >> From: Brian Granger >> Subject: Re: [IPython-dev] changing working directory from IPython >> server >> To: IPython developers list >> Message-ID: >> >> >> Content-Type: text/plain; charset=UTF-8 >> >> Oh, I might have understood. >> >> Mark can you clarify what exactly you are seeing and what you expect to >> see? >> >> On Mon, Aug 11, 2014 at 10:03 AM, Thomas Kluyver wrote: >> > On 11 August 2014 07:51, Brian Granger wrote: >> >> >> >> I don't remember that we made the decision to have this behavior. It >> >> is possibly I just don't remember, but today I would consider that to >> >> be a bug. Mark, could you open a GitHub issue on this to trigger a >> >> discussion. >> > >> > >> > I'm pretty sure that what Mark describes is how we expect it to work - >> > when >> > you upload a file, the server doesn't know where in your filesystem that >> > data is coming from, so it can't switch to that directory and use the >> > original notebook there, which I think is what Mark wants. The >> > disconnect is >> > that the UI implicitly encourages uploading, although that doesn't >> > really >> > make sense when you're running the server as a local application. >> > >> > Thomas >> > >> - >> Brian E. Granger >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From ksmith at enthought.com Mon Aug 11 16:38:12 2014 From: ksmith at enthought.com (Kurt Smith) Date: Mon, 11 Aug 2014 15:38:12 -0500 Subject: [IPython-dev] ANN: DistArray 0.5 release Message-ID: ====================================== DistArray 0.5 release ====================================== **Mailing list:** distarray at googlegroups.com **Documentation:** http://distarray.readthedocs.org **License:** Three-clause BSD **Python versions:** 2.7, 3.3, and 3.4 **OS support:** \*nix and Mac OS X What is DistArray? ------------------ DistArray aims to bring the ease-of-use of NumPy to data-parallel high-performance computing. It provides distributed multi-dimensional NumPy arrays, distributed ufuncs, and distributed IO capabilities. It can efficiently interoperate with external distributed libraries like Trilinos. DistArray works with NumPy and builds on top of it in a flexible and natural way. 0.5 Release ----------- Noteworthy improvements in this release include: * closer alignment with NumPy's API, * support for Python 3.4 (existing support for Python 2.7 and 3.3), * a performance-oriented MPI-only mode for deployment on clusters and supercomputers, * a way to register user-defined functions to be callable locally on worker processes, * more consistent naming of sub-packages, * testing with MPICH2 (already tested against OpenMPI), * improved and expanded examples, * installed version testable via ``distarray.test()``, and * performance and scaling improvements. With this release, DistArray ready for real-world testing and deployment. The project is still evolving rapidly and we appreciate the continued input from the larger scientific-Python community. Existing features ----------------- DistArray: * supports NumPy-like slicing, reductions, and ufuncs on distributed multidimensional arrays; * has a client-engine process design -- data resides on the worker processes, commands are initiated from master; * allows full control over what is executed on the worker processes and integrates transparently with the master process; * allows direct communication between workers, bypassing the master process for scalability; * integrates with IPython.parallel for interactive creation and exploration of distributed data; * supports distributed ufuncs (currently without broadcasting); * builds on and leverages MPI via MPI4Py in a transparent and user-friendly way; * has basic support for unstructured arrays; * supports user-controllable array distributions across workers (block, cyclic, block-cyclic, and unstructured) on a per-axis basis; * has a straightforward API to control how an array is distributed; * has basic plotting support for visualization of array distributions; * separates the array?s distribution from the array?s data -- useful for slicing, reductions, redistribution, broadcasting, and other operations; * implements distributed random arrays; * supports ``.npy``-like flat-file IO and hdf5 parallel IO (via ``h5py``); leverages MPI-based IO parallelism in an easy-to-use and transparent way; and * supports the distributed array protocol [protocol]_, which allows independently developed parallel libraries to share distributed arrays without copying, analogous to the PEP-3118 new buffer protocol. Planned features and roadmap ---------------------------- Near-term features and improvements include: * array re-distribution capabilities; * lazy evaluation and deferred computation for latency hiding; * interoperation with Trilinos [Trilinos]_; and * distributed broadcasting support. The longer-term roadmap includes: * Integration with other packages [petsc]_ that subscribe to the distributed array protocol [protocol]_; * Distributed fancy indexing; * Out-of-core computations; * Support for distributed sorting and other non-trivial distributed algorithms; and * End-user control over communication and temporary array creation, and other performance aspects of distributed computations. History and funding ------------------- Brian Granger started DistArray as a NASA-funded SBIR project in 2008. Enthought picked it up as part of a DOE Phase II SBIR [SBIR]_ to provide a generally useful distributed array package. It builds on NumPy, MPI, MPI4Py, IPython, IPython.parallel, and interfaces with the Trilinos suite of distributed HPC solvers (via PyTrilinos [Trilinos]_). This material is based upon work supported by the Department of Energy under Award Number DE-SC0007699. This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof. .. [protocol] http://distributed-array-protocol.readthedocs.org/en/rel-0.10.0/ .. [Trilinos] http://trilinos.org/ .. [petsc] http://www.mcs.anl.gov/petsc/ .. [SBIR] http://www.sbir.gov/sbirsearch/detail/410257 -- Kurt W. Smith, Ph.D. Enthought, Inc. | 512.536.1057 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Mon Aug 11 16:47:15 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 11 Aug 2014 13:47:15 -0700 Subject: [IPython-dev] ANN: DistArray 0.5 release In-Reply-To: References: Message-ID: Kurt, Thanks for sharing, this looks great! Cheers, Brian On Mon, Aug 11, 2014 at 1:38 PM, Kurt Smith wrote: > ====================================== > DistArray 0.5 release > ====================================== > > **Mailing list:** distarray at googlegroups.com > > **Documentation:** http://distarray.readthedocs.org > > **License:** Three-clause BSD > > **Python versions:** 2.7, 3.3, and 3.4 > > **OS support:** \*nix and Mac OS X > > What is DistArray? > ------------------ > > DistArray aims to bring the ease-of-use of NumPy to data-parallel > high-performance computing. It provides distributed multi-dimensional NumPy > arrays, distributed ufuncs, and distributed IO capabilities. It can > efficiently interoperate with external distributed libraries like Trilinos. > DistArray works with NumPy and builds on top of it in a flexible and natural > way. > > 0.5 Release > ----------- > > Noteworthy improvements in this release include: > > * closer alignment with NumPy's API, > * support for Python 3.4 (existing support for Python 2.7 and 3.3), > * a performance-oriented MPI-only mode for deployment on clusters and > supercomputers, > * a way to register user-defined functions to be callable locally on worker > processes, > * more consistent naming of sub-packages, > * testing with MPICH2 (already tested against OpenMPI), > * improved and expanded examples, > * installed version testable via ``distarray.test()``, and > * performance and scaling improvements. > > With this release, DistArray ready for real-world testing and deployment. > The > project is still evolving rapidly and we appreciate the continued input from > the larger scientific-Python community. > > Existing features > ----------------- > > DistArray: > > * supports NumPy-like slicing, reductions, and ufuncs on distributed > multidimensional arrays; > * has a client-engine process design -- data resides on the worker > processes, > commands are initiated from master; > * allows full control over what is executed on the worker processes and > integrates transparently with the master process; > * allows direct communication between workers, bypassing the master process > for scalability; > * integrates with IPython.parallel for interactive creation and exploration > of > distributed data; > * supports distributed ufuncs (currently without broadcasting); > * builds on and leverages MPI via MPI4Py in a transparent and user-friendly > way; > * has basic support for unstructured arrays; > * supports user-controllable array distributions across workers (block, > cyclic, block-cyclic, and unstructured) on a per-axis basis; > * has a straightforward API to control how an array is distributed; > * has basic plotting support for visualization of array distributions; > * separates the array?s distribution from the array?s data -- useful for > slicing, reductions, redistribution, broadcasting, and other operations; > * implements distributed random arrays; > * supports ``.npy``-like flat-file IO and hdf5 parallel IO (via ``h5py``); > leverages MPI-based IO parallelism in an easy-to-use and transparent way; > and > * supports the distributed array protocol [protocol]_, which allows > independently developed parallel libraries to share distributed arrays > without copying, analogous to the PEP-3118 new buffer protocol. > > Planned features and roadmap > ---------------------------- > > Near-term features and improvements include: > > * array re-distribution capabilities; > * lazy evaluation and deferred computation for latency hiding; > * interoperation with Trilinos [Trilinos]_; and > * distributed broadcasting support. > > The longer-term roadmap includes: > > * Integration with other packages [petsc]_ that subscribe to the distributed > array protocol [protocol]_; > * Distributed fancy indexing; > * Out-of-core computations; > * Support for distributed sorting and other non-trivial distributed > algorithms; and > * End-user control over communication and temporary array creation, and > other > performance aspects of distributed computations. > > History and funding > ------------------- > > Brian Granger started DistArray as a NASA-funded SBIR project in 2008. > Enthought picked it up as part of a DOE Phase II SBIR [SBIR]_ to provide a > generally useful distributed array package. It builds on NumPy, MPI, > MPI4Py, > IPython, IPython.parallel, and interfaces with the Trilinos suite of > distributed HPC solvers (via PyTrilinos [Trilinos]_). > > This material is based upon work supported by the Department of Energy under > Award Number DE-SC0007699. > > This report was prepared as an account of work sponsored by an agency of the > United States Government. Neither the United States Government nor any > agency > thereof, nor any of their employees, makes any warranty, express or implied, > or assumes any legal liability or responsibility for the accuracy, > completeness, or usefulness of any information, apparatus, product, or > process > disclosed, or represents that its use would not infringe privately owned > rights. Reference herein to any specific commercial product, process, or > service by trade name, trademark, manufacturer, or otherwise does not > necessarily constitute or imply its endorsement, recommendation, or favoring > by the United States Government or any agency thereof. The views and > opinions > of authors expressed herein do not necessarily state or reflect those of the > United States Government or any agency thereof. > > > .. [protocol] > http://distributed-array-protocol.readthedocs.org/en/rel-0.10.0/ > .. [Trilinos] http://trilinos.org/ > .. [petsc] http://www.mcs.anl.gov/petsc/ > .. [SBIR] http://www.sbir.gov/sbirsearch/detail/410257 > > -- > Kurt W. Smith, Ph.D. > Enthought, Inc. | 512.536.1057 > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From markbak at gmail.com Mon Aug 11 16:56:19 2014 From: markbak at gmail.com (Mark Bakker) Date: Mon, 11 Aug 2014 22:56:19 +0200 Subject: [IPython-dev] changing working directory from IPython server Message-ID: Sorry to keep pestering you with this, Brian. I explored it some more, and narrowed it down to the following. Let's say my notebook is in /mark/notebooks When I start ipython notebook in /mark, I can nicely click on the directory notebooks on the dashboard and open the notebook I want. If, on the other hand, I start ipython notebook in /mark/Documents. Then there is no way to 'surf' to the correct /mark/notebooks, as there is no cd.. place to click to go back to /mark and then to /mark/notebooks. When I do the 'To import a notebook, drag the file onto the listing below or *click here*' option, it only let's me 'upload' the selected notebook, which means copying to the directory where I started ipython notebook. The Anaconda guys run into exactly this problem, as their nice Notebook icon starts in /user/Documents/IPython_Notebooks, and you can only surf to a directory lower than that. So what I think we need is either a way to surf to a higher directory, or to have the option to 'click here' as it is now but then not being forced to 'upload' the notebook. Mark From: Brian Granger > Subject: Re: [IPython-dev] changing working directory from IPython > server > To: IPython developers list > Message-ID: > UdrTukpheV2hqcJeebH-zzuPdQ4wQ at mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > Mark, > > OK I think I understand, and I don't consider this a bug. If the > notebook you are starting out with is on the same machine, there is no > reason to "upload" that notebook. Just navigate the notebook dashboard > to the directory where that notebook already is and open it. Is there > some reason you can't do that and are instead trying to upload the > notebook. > > Cheers, > > Brian > > On Mon, Aug 11, 2014 at 1:11 PM, Mark Bakker wrote: > > Thanks for considering this issue. > > > > The behavior now: > > I type: ipython notebook to start a notebook. > > At the dashboard, I click on 'To import a notebook, drag the file onto > the > > listing below or *click here*. When I click there, a window pops up, I > go to > > the directory with my notebook file, click on it, and then it shows up in > > the list of files, with next to it a blue Upload button. When I click on > the > > Upload button the notebook is copied to the directory where I started the > > IPython Notebook. > > > > The problem: The notebook gets copied to the directory where I started > > ipython notebook. That is not very useful, as all associated data files > are > > still in the directory where the notebook originated. > > > > Does that make it clear? Should I file a bug report? > > > > Mark > > > > ps. It seems that the Anaconda guys are also expecting the behavior of > the > > Notebook to change. They now have a useful 'start notebook' icon, but > that > > starts in 'Documents/IPython Notebooks. And that faces the same problem > as I > > described above. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Mon Aug 11 17:10:35 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 11 Aug 2014 17:10:35 -0400 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: <88388EF9-C204-4E06-8003-6E4BE94708EA@gmail.com> For security reasons we can't allow the notebook to browse to directories higher than it's starting directory. Thus, the only solution is to start the notebook from the highest directory you want to access notebooks from. Given how the notebook works, I think that anaconda has made a less than ideal choice by not starting the notebook server in the users home directory. Cheers, Brian Sent from my iPhone > On Aug 11, 2014, at 4:56 PM, Mark Bakker wrote: > > Sorry to keep pestering you with this, Brian. > > I explored it some more, and narrowed it down to the following. > > Let's say my notebook is in /mark/notebooks > > When I start ipython notebook in /mark, I can nicely click on the directory notebooks on the dashboard and open the notebook I want. > > If, on the other hand, I start ipython notebook in /mark/Documents. Then there is no way to 'surf' to the correct /mark/notebooks, as there is no cd.. place to click to go back to /mark and then to /mark/notebooks. When I do the 'To import a notebook, drag the file onto the listing below or *click here*' option, it only let's me 'upload' the selected notebook, which means copying to the directory where I started ipython notebook. > > The Anaconda guys run into exactly this problem, as their nice Notebook icon starts in /user/Documents/IPython_Notebooks, and you can only surf to a directory lower than that. > > So what I think we need is either a way to surf to a higher directory, or to have the option to 'click here' as it is now but then not being forced to 'upload' the notebook. > > Mark > > >> From: Brian Granger >> Subject: Re: [IPython-dev] changing working directory from IPython >> server >> To: IPython developers list >> Message-ID: >> >> Content-Type: text/plain; charset=UTF-8 >> >> Mark, >> >> OK I think I understand, and I don't consider this a bug. If the >> notebook you are starting out with is on the same machine, there is no >> reason to "upload" that notebook. Just navigate the notebook dashboard >> to the directory where that notebook already is and open it. Is there >> some reason you can't do that and are instead trying to upload the >> notebook. >> >> Cheers, >> >> Brian >> >> On Mon, Aug 11, 2014 at 1:11 PM, Mark Bakker wrote: >> > Thanks for considering this issue. >> > >> > The behavior now: >> > I type: ipython notebook to start a notebook. >> > At the dashboard, I click on 'To import a notebook, drag the file onto the >> > listing below or *click here*. When I click there, a window pops up, I go to >> > the directory with my notebook file, click on it, and then it shows up in >> > the list of files, with next to it a blue Upload button. When I click on the >> > Upload button the notebook is copied to the directory where I started the >> > IPython Notebook. >> > >> > The problem: The notebook gets copied to the directory where I started >> > ipython notebook. That is not very useful, as all associated data files are >> > still in the directory where the notebook originated. >> > >> > Does that make it clear? Should I file a bug report? >> > >> > Mark >> > >> > ps. It seems that the Anaconda guys are also expecting the behavior of the >> > Notebook to change. They now have a useful 'start notebook' icon, but that >> > starts in 'Documents/IPython Notebooks. And that faces the same problem as I >> > described above. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Mon Aug 11 18:34:49 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 11 Aug 2014 15:34:49 -0700 Subject: [IPython-dev] ANN: DistArray 0.5 release In-Reply-To: References: Message-ID: On Mon, Aug 11, 2014 at 1:47 PM, Brian Granger wrote: > Thanks for sharing, this looks great! Indeed, and I'm very happy to see that IPython has made a contribution to distarray :) Great job, Kurt & team! f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Mon Aug 11 18:38:09 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 11 Aug 2014 15:38:09 -0700 Subject: [IPython-dev] IPython.parallel slow push In-Reply-To: References: Message-ID: On Mon, Aug 11, 2014 at 6:56 AM, Wes Turner wrote: > This [2] seems to suggest that anything that isn't a buffer, > str/bytes, or numpy array is pickled and copied. > That is indeed correct. > Would it be faster to ETL into something like HDF5 (e.g. w/ > Pandas/PyTables) and just synchronize the dataset URI? > Absolutely. IPython.parallel is NOT the right tool to use to move large amounts of data around between machines. It's an important problem in parallel/distributed computing, but also a very challenging one that is beyond our scope and resources. When using IPython.parallel, you should think of it as a good way to - coordinate computation - move code around - move *small* data around - have interactive control in parallel settings But you should have a non-IPython strategy for moving big chunks of data around. The right answer to that question will vary from one context to another. In some cases a simple NFS mount may be enough, elsewhere something like Hadoop FS or Disco FS may work, or a well-sharded database, or whatever. But it's simply a problem that we consider orthogonal to what IPython.parallel can do well. Hope this helps, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Aug 11 18:47:01 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 11 Aug 2014 15:47:01 -0700 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: On 11 August 2014 13:56, Mark Bakker wrote: > So what I think we need is either a way to surf to a higher directory, or > to have the option to 'click here' as it is now but then not being forced > to 'upload' the notebook. > Allow me to plug nbopen again: it's a tool to open notebooks. If there's a notebook server running in any directory above where that notebook is, it will use that. If not, it will start a new notebook server which can open that. So there's no need to think about where you're launching the notebook. It works from the command line on all platforms, and from GUI file managers on open source platforms (pull requests for other platforms welcome). https://github.com/takluyver/nbopen Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From cyrille.rossant at gmail.com Tue Aug 12 00:16:00 2014 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Tue, 12 Aug 2014 09:16:00 +0500 Subject: [IPython-dev] IPython event loop integration In-Reply-To: References: Message-ID: Hi all, We're currently implementing an IPython notebook backend for Vispy. We're having a hard time figuring out how to integrate timer-based animations in the notebook [1]. We'd really appreciate if one of you could have a look at our problem. This is the last step of our GSoC student's project, which terminates in a few days. Many thanks, Cyrille [1] https://groups.google.com/forum/#!topic/vispy-dev/blh2odeMgvI -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.hirschfeld at gmail.com Tue Aug 12 03:51:21 2014 From: dave.hirschfeld at gmail.com (Dave Hirschfeld) Date: Tue, 12 Aug 2014 07:51:21 +0000 (UTC) Subject: [IPython-dev] changing working directory from IPython server References: Message-ID: Mark Bakker gmail.com> writes: > > > Sorry to keep pestering you with this, Brian. > > I explored it some more, and narrowed it down to the following. > > Let's say my notebook is in /mark/notebooks > > > When I start ipython notebook in /mark, I can nicely click on the directory notebooks on the dashboard and open the notebook I want. > > If, on the other hand, I start ipython notebook in /mark/Documents. Then there is no way to 'surf' to the correct /mark/notebooks, as there is no cd.. place to click to go back to /mark and then to /mark/notebooks. When I do the ?'To import a notebook, drag the file onto the listing below or *click here*' option, it only let's me 'upload' the selected notebook, which means copying to the directory where I started ipython notebook. > > > So what I think we need is either a way to surf to a higher directory, or to have the option to 'click here' as it is now but then not being forced to 'upload' the notebook. > > Mark > One solution if you have folders in different locations would be to simlink them (mklink /J in windows) into the notebook server directory HTH, Dave From markbak at gmail.com Tue Aug 12 04:16:23 2014 From: markbak at gmail.com (Mark Bakker) Date: Tue, 12 Aug 2014 10:16:23 +0200 Subject: [IPython-dev] changing working directory from IPython server Message-ID: Brian, All right, so browsing to a higher directory is out. What about the 'uploading' feature. Would it not be possible to use the 'To import a notebook, drag the file onto the listing below or *click here*' option and then not be forced to upload the notebook? Or are there security issues with that as well? Mark ps. I hope the Anaconda guys are reading this and will consider changing their default starting location, as my students really like their clickable icon to start ipython notebooks. From: Brian Granger > Subject: Re: [IPython-dev] changing working directory from IPython > server > To: IPython developers list > Message-ID: <88388EF9-C204-4E06-8003-6E4BE94708EA at gmail.com> > Content-Type: text/plain; charset="us-ascii" > > For security reasons we can't allow the notebook to browse to directories > higher than it's starting directory. Thus, the only solution is to start > the notebook from the highest directory you want to access notebooks from. > > Given how the notebook works, I think that anaconda has made a less than > ideal choice by not starting the notebook server in the users home > directory. > > Cheers, > > Brian > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moritz.beber at gmail.com Tue Aug 12 06:31:15 2014 From: moritz.beber at gmail.com (Moritz Beber) Date: Tue, 12 Aug 2014 12:31:15 +0200 Subject: [IPython-dev] IPython.parallel slow push In-Reply-To: References: Message-ID: On Tue, Aug 12, 2014 at 12:38 AM, Fernando Perez wrote: > On Mon, Aug 11, 2014 at 6:56 AM, Wes Turner wrote: > >> This [2] seems to suggest that anything that isn't a buffer, >> str/bytes, or numpy array is pickled and copied. >> > > That is indeed correct. > > >> Would it be faster to ETL into something like HDF5 (e.g. w/ >> Pandas/PyTables) and just synchronize the dataset URI? >> > > Absolutely. > > IPython.parallel is NOT the right tool to use to move large amounts of > data around between machines. It's an important problem in > parallel/distributed computing, but also a very challenging one that is > beyond our scope and resources. > As I said, I didn't move anything between machines, just locally. Still it uses ZMQ and I get that IPython is not meant to handle this situation. Simply using a shelve (relying on pickle here) and loading the contents in each kernel already improved the time needed a lot. > > When using IPython.parallel, you should think of it as a good way to > > - coordinate computation > - move code around > - move *small* data around > - have interactive control in parallel settings > > But you should have a non-IPython strategy for moving big chunks of data > around. The right answer to that question will vary from one context to > another. In some cases a simple NFS mount may be enough, elsewhere > something like Hadoop FS or Disco FS may work, or a well-sharded database, > or whatever. > > But it's simply a problem that we consider orthogonal to what > IPython.parallel can do well. > > Hope this helps, > > f > > Thank you for your input. -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Aug 12 12:36:25 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 12 Aug 2014 09:36:25 -0700 Subject: [IPython-dev] IPython event loop integration In-Reply-To: References: Message-ID: Hi Cyrille, Having a quick scan through the thread you linked to, I wonder if it's possible to use the kernel event loop to schedule update events. pyzmq subclasses tornado's event loop, and provides a DelayedCallback class [1], and tornado itself has a PeriodicCallback class [2]. [1] http://zeromq.github.io/pyzmq/api/generated/zmq.eventloop.ioloop.html#delayedcallback [2] http://tornado.readthedocs.org/en/latest/ioloop.html#tornado.ioloop.PeriodicCallback Thomas On 11 August 2014 21:16, Cyrille Rossant wrote: > Hi all, > > We're currently implementing an IPython notebook backend for Vispy. > We're having a hard time figuring out how to integrate timer-based > animations in the notebook [1]. We'd really appreciate if one of you > could have a look at our problem. This is the last step of our GSoC > student's project, which terminates in a few days. > > Many thanks, > Cyrille > > [1] https://groups.google.com/forum/#!topic/vispy-dev/blh2odeMgvI > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Aug 12 12:38:54 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 12 Aug 2014 09:38:54 -0700 Subject: [IPython-dev] changing working directory from IPython server In-Reply-To: References: Message-ID: On 12 August 2014 01:16, Mark Bakker wrote: > 'To import a notebook, drag the file onto the listing below or *click > here*' option and then not be forced to upload the notebook? Or are there > security issues with that as well? As far as I know, the upload interface sees only the contents of the file, not the location, so it's impossible to work with it on disk from its original location based on pulling it into the browser. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From shipman.william at gmail.com Tue Aug 12 12:43:11 2014 From: shipman.william at gmail.com (William Shipman) Date: Tue, 12 Aug 2014 18:43:11 +0200 Subject: [IPython-dev] Fwd: Using DirectView.execute to import modules on worker processes In-Reply-To: References: Message-ID: I have looked through the documentation about importing modules on IPython Parallel worker processes, and it seems imports of the form "import foo as bar" are not supported. However, I found that using DirectView.execute allows me to get around this. Is this safe, if not what can go wrong? Here's the code I'm using to import part of the dateutil library and : from IPython.parallel import Client ipython_client = Client() dview = ipython_client.direct_view() dview.execute('from dateutil import parser as dtparser') I am only using on my local PC, not a cluster. After executing the above I can happily run code that uses dtparser using dview.map_sync. I can rewrite my code to avoid doing this but I'd like to know why I shouldn't use this method. Thanks in advance. P.S. I posted this to IPython-User but that mailing list seems not to be active. I anyone is receiving this for a second time, I apologize. -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Aug 12 12:47:20 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 12 Aug 2014 09:47:20 -0700 Subject: [IPython-dev] Fwd: Using DirectView.execute to import modules on worker processes In-Reply-To: References: Message-ID: Hi William, I don't think there should be any problem with using DirectView.execute() for this. Min, who's the expert on IPython.parallel, is away at the moment, but he should be able to give you a proper answer when he gets back in a couple of days. Thanks, Thomas On 12 August 2014 09:43, William Shipman wrote: > I have looked through the documentation about importing modules on IPython > Parallel worker processes, and it seems imports of the form "import foo as > bar" are not supported. However, I found that using DirectView.execute > allows me to get around this. Is this safe, if not what can go wrong? > > Here's the code I'm using to import part of the dateutil library and : > > from IPython.parallel import Client > ipython_client = Client() > dview = ipython_client.direct_view() > dview.execute('from dateutil import parser as dtparser') > > I am only using on my local PC, not a cluster. After executing the above > I can happily run code that uses dtparser using dview.map_sync. I can > rewrite my code to avoid doing this but I'd like to know why I shouldn't > use this method. > > Thanks in advance. > > P.S. I posted this to IPython-User but that mailing list seems not to be > active. I anyone is receiving this for a second time, I apologize. > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shipman.william at gmail.com Tue Aug 12 13:03:50 2014 From: shipman.william at gmail.com (William Shipman) Date: Tue, 12 Aug 2014 19:03:50 +0200 Subject: [IPython-dev] Fwd: Using DirectView.execute to import modules on worker processes In-Reply-To: References: Message-ID: Thanks for the quick reply. Should I post a message again next week once Min is available again? I guess IPython-User should not be used anymore? Thanks, William. On 12 August 2014 18:47, Thomas Kluyver wrote: > Hi William, > > I don't think there should be any problem with using DirectView.execute() > for this. Min, who's the expert on IPython.parallel, is away at the moment, > but he should be able to give you a proper answer when he gets back in a > couple of days. > > Thanks, > Thomas > > > On 12 August 2014 09:43, William Shipman > wrote: > >> I have looked through the documentation about importing modules on >> IPython Parallel worker processes, and it seems imports of the form "import >> foo as bar" are not supported. However, I found that using >> DirectView.execute allows me to get around this. Is this safe, if not what >> can go wrong? >> >> Here's the code I'm using to import part of the dateutil library and : >> >> from IPython.parallel import Client >> ipython_client = Client() >> dview = ipython_client.direct_view() >> dview.execute('from dateutil import parser as dtparser') >> >> I am only using on my local PC, not a cluster. After executing the above >> I can happily run code that uses dtparser using dview.map_sync. I can >> rewrite my code to avoid doing this but I'd like to know why I shouldn't >> use this method. >> >> Thanks in advance. >> >> P.S. I posted this to IPython-User but that mailing list seems not to be >> active. I anyone is receiving this for a second time, I apologize. >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug.blank at gmail.com Tue Aug 12 13:09:58 2014 From: doug.blank at gmail.com (Doug Blank) Date: Tue, 12 Aug 2014 13:09:58 -0400 Subject: [IPython-dev] Kernel-independent magics? Message-ID: We are beginning to take our Pure-Python languages out from Calico and implement them as stand-alone kernels (based on KernelBase) for Jupyter (IPython 3). This includes Scheme, Basic, Logo, Little Computer (LC3, assembly language), etc. Wondering about how magics might be handled in Jupyter (In the Calico Kernel, we handle magics in C# before passing the rest of the code to the language). Currently, it looks like magics are handled in IPython by the Interactive Python Shell. There are many magics that are language independent and could be handled more abstractly (via JavaScript or server): * %%latex * %%html * %connect_info * %%file - create a file, with cell contents * %%time - give execution time (not timeit, which figures out how many times to run) * %logstart * many others In addition, there is the need to allow others to write magics for specific kernels, or for general kernels. That would imply some kind of %load_ext. Also, there is generic ! shell execution. Are there any plans on how to handle these functions in a kernel-independent manner? If not, it would be good for kernel authors to adopt some standards for magics. Thanks for any pointers or comments! -Doug -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Aug 12 13:20:48 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 12 Aug 2014 10:20:48 -0700 Subject: [IPython-dev] Fwd: Using DirectView.execute to import modules on worker processes In-Reply-To: References: Message-ID: On 12 August 2014 10:03, William Shipman wrote: > Thanks for the quick reply. Should I post a message again next week once > Min is available again? > I'll try to point him to this thread, but if you don't hear from him by next week, then ping the list again, or drop into the help chat room at http://www.hipchat.com/ghSp7E1uY > I guess IPython-User should not be used anymore? > We're gently phasing it out so there's one list for conversations - we don't completely ignore -user, but yes, posting to -dev is preferred. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Aug 12 13:32:41 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 12 Aug 2014 10:32:41 -0700 Subject: [IPython-dev] Kernel-independent magics? In-Reply-To: References: Message-ID: Our take on this is that IPython's magic functions are a feature of the IPython kernel. This kernel runs code written in 'IPython syntax', which is a loosely specified superset of Python code, including some features designed for convenient interactive use (%magics, !shell commands, help?, etc.). If other kernels want to implement extra syntax for their own languages to aid interactive use, that's up to them, but we don't expect them to, and we're not trying to standardise those features. The same goes for IPython (kernel side) extensions: if other kernels need an extension system, they should implement that using the tools and conventions that make sense for that language. We do, however, intend to build tools to manage pure client side notebook extensions in a kernel-independent way. Thomas On 12 August 2014 10:09, Doug Blank wrote: > We are beginning to take our Pure-Python languages out from Calico and > implement them as stand-alone kernels (based on KernelBase) for Jupyter > (IPython 3). This includes Scheme, Basic, Logo, Little Computer (LC3, > assembly language), etc. > > Wondering about how magics might be handled in Jupyter (In the Calico > Kernel, we handle magics in C# before passing the rest of the code to the > language). Currently, it looks like magics are handled in IPython by the > Interactive Python Shell. There are many magics that are language > independent and could be handled more abstractly (via JavaScript or server): > > * %%latex > * %%html > * %connect_info > * %%file - create a file, with cell contents > * %%time - give execution time (not timeit, which figures out how many > times to run) > * %logstart > * many others > > In addition, there is the need to allow others to write magics for > specific kernels, or for general kernels. That would imply some kind of > %load_ext. > > Also, there is generic ! shell execution. > > Are there any plans on how to handle these functions in a > kernel-independent manner? If not, it would be good for kernel authors to > adopt some standards for magics. > > Thanks for any pointers or comments! > > -Doug > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug.blank at gmail.com Tue Aug 12 14:04:35 2014 From: doug.blank at gmail.com (Doug Blank) Date: Tue, 12 Aug 2014 14:04:35 -0400 Subject: [IPython-dev] Kernel-independent magics? In-Reply-To: References: Message-ID: On Tue, Aug 12, 2014 at 1:32 PM, Thomas Kluyver wrote: > Our take on this is that IPython's magic functions are a feature of the > IPython kernel. This kernel runs code written in 'IPython syntax', which is > a loosely specified superset of Python code, including some features > designed for convenient interactive use (%magics, !shell commands, help?, > etc.). If other kernels want to implement extra syntax for their own > languages to aid interactive use, that's up to them, but we don't expect > them to, and we're not trying to standardise those features. > Ok, thanks for the quick response. That makes sense as many IPython magics are intricately interwoven with Python. I think that there is a role for a simplified, non-interwoven, standard set of magics. Ok, we'll port our C# magic handler to Python, and share that for use with those extending the KernelBase. Then a Python-based kernel can easily get some nice/simple meta commands. > The same goes for IPython (kernel side) extensions: if other kernels need > an extension system, they should implement that using the tools and > conventions that make sense for that language. We do, however, intend to > build tools to manage pure client side notebook extensions in a > kernel-independent way. > Sounds good! That should help on the front end. -Doug > > Thomas > > > On 12 August 2014 10:09, Doug Blank wrote: > >> We are beginning to take our Pure-Python languages out from Calico and >> implement them as stand-alone kernels (based on KernelBase) for Jupyter >> (IPython 3). This includes Scheme, Basic, Logo, Little Computer (LC3, >> assembly language), etc. >> >> Wondering about how magics might be handled in Jupyter (In the Calico >> Kernel, we handle magics in C# before passing the rest of the code to the >> language). Currently, it looks like magics are handled in IPython by the >> Interactive Python Shell. There are many magics that are language >> independent and could be handled more abstractly (via JavaScript or server): >> >> * %%latex >> * %%html >> * %connect_info >> * %%file - create a file, with cell contents >> * %%time - give execution time (not timeit, which figures out how many >> times to run) >> * %logstart >> * many others >> >> In addition, there is the need to allow others to write magics for >> specific kernels, or for general kernels. That would imply some kind of >> %load_ext. >> >> Also, there is generic ! shell execution. >> >> Are there any plans on how to handle these functions in a >> kernel-independent manner? If not, it would be good for kernel authors to >> adopt some standards for magics. >> >> Thanks for any pointers or comments! >> >> -Doug >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Aug 13 02:47:08 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 12 Aug 2014 23:47:08 -0700 Subject: [IPython-dev] IPython.parallel slow push In-Reply-To: References: Message-ID: On Tue, Aug 12, 2014 at 3:31 AM, Moritz Beber wrote: > As I said, I didn't move anything between machines, just locally. Sorry, when I said machines I should have said 'engines'. It doesn't matter that the engines are on the same physical node or not, the data is still transferred between processes, and that's what was killing you. Sorry for the confusion. Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Wed Aug 13 12:02:26 2014 From: benjaminrk at gmail.com (Min RK) Date: Wed, 13 Aug 2014 09:02:26 -0700 Subject: [IPython-dev] IPython event loop integration In-Reply-To: References: Message-ID: <19D5DC10-6002-40BB-99B0-49364E4FD822@gmail.com> Tornado's PeriodicCallback ought to be the way to go for timers in an IPython kernel. There may be some issues if matplotlib is involved, setting up its own eventloop. In that case, you may want to use matplotlib's own Timer objects. An advantage of mpl timer objects is that they will work in all sorts of cases, not just tornado-based IPython. -MinRK > On Aug 12, 2014, at 9:36, Thomas Kluyver wrote: > > Hi Cyrille, > > Having a quick scan through the thread you linked to, I wonder if it's possible to use the kernel event loop to schedule update events. pyzmq subclasses tornado's event loop, and provides a DelayedCallback class [1], and tornado itself has a PeriodicCallback class [2]. > > [1] http://zeromq.github.io/pyzmq/api/generated/zmq.eventloop.ioloop.html#delayedcallback > [2] http://tornado.readthedocs.org/en/latest/ioloop.html#tornado.ioloop.PeriodicCallback > > Thomas > > >> On 11 August 2014 21:16, Cyrille Rossant wrote: >> Hi all, >> >> We're currently implementing an IPython notebook backend for Vispy. >> We're having a hard time figuring out how to integrate timer-based >> animations in the notebook [1]. We'd really appreciate if one of you >> could have a look at our problem. This is the last step of our GSoC >> student's project, which terminates in a few days. >> >> Many thanks, >> Cyrille >> >> [1] https://groups.google.com/forum/#!topic/vispy-dev/blh2odeMgvI >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Wed Aug 13 13:17:29 2014 From: benjaminrk at gmail.com (MinRK) Date: Wed, 13 Aug 2014 10:17:29 -0700 Subject: [IPython-dev] mathjax equation number is incremented on cntr-enter In-Reply-To: <53E45619.4040302@gmail.com> References: <53E45619.4040302@gmail.com> Message-ID: This is a known issue , where mathjax generates an equation number for each equation it renders, so they keep increasing on each render. In order to get proper equation numbering, all equations (markdown cells) on the page would need to be re-rendered every time any equation is. There isn?t a good shortcut for this at the moment, but reloading the page should work. -MinRK ? On Thu, Aug 7, 2014 at 9:46 PM, Zolt?n V?r?s wrote: > Hi all, > > I don't know, whether this is a known issue (a quick search did not turn > up anything), in fact, I don't even know, whether this is an issue at all, > but I believe, the markdown cell did behave differently in the past. In > short, it seems to me that mathjax increments the equation number, whenever > I press Cntr-Enter on a markdown cell. Here is my notebook, and here is the > code that produces the problem: > > \begin{equation} > x = y > \end{equation} > > I haven't removed anything, yet, the equation is labelled (4). Can someone > comment on this? In particular, whether this was intended? Is this, > perhaps, related to the security problem that was floating around recently? > I am running the latest code from master. > > Cheers, > > Zolt?n > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Wed Aug 13 13:26:22 2014 From: benjaminrk at gmail.com (MinRK) Date: Wed, 13 Aug 2014 10:26:22 -0700 Subject: [IPython-dev] Separating the notebook from the file manager+text editor+terminal In-Reply-To: References: Message-ID: I think this sounds like a good plan. I?m not 100% sure how to go about implementing it, though. It would require that our webapp code be a bit more modular. We took some steps in that direction with 2.0 - Brian and Zach moved each component into discrete subpackages and URL areas, and are loaded with a single call to load_handlers. We could extend this a bit further to something like load_component, and figure out a way to configure / register available components. -MinRK ? On Sun, Aug 10, 2014 at 5:22 PM, Thomas Kluyver wrote: > We've often said that, for people accessing the notebook on a remote > server, we need to provide certain basic utilities - like the dashboard, > which is now under pressure to evolve into a file manager, and an > in-browser terminal and text editor, both of which we intend to add to the > notebook. > > I propose that we separate development and packaging of the notebook from > the project to assemble this trio. The new project would live under the > Jupyter org, and be called something like 'utility-apps'. I see several > advantages of this: > > - A different crowd of developers may be attracted to work on that, > without having to know anything about the internals of IPython's machinery > for handling and executing code. For instance, I'm pushing back on turning > the dashboard into a full blown file manager, because I think that adds a > lot of extra complexity in a codebase that's already complex, but if it was > a separate project, it could grow into a file manager without being a > problem. > - People running the notebook locally may want to install just the > notebook component, and not poor web-based clones of applications which > they have well developed equivalents for already installed. > - There are lots of potential users of the file manager/text > editor/terminal trio who probably wouldn't use the notebook: for instance, > it could be a basic admin interface to any server. > - It could become a base on which people could write other HTML > applications to meet their own needs. Following our usual MO, I wouldn't > try to design it like this straight away, but keep it in mind, and evolve > the APIs as things mature. > > There are still plenty of questions to be answered. How would this new > component interface with the notebook server? What would the notebook look > like without the file manager installed? But these could certainly be > worked out, and I think they're worth working out in order to keep the > scope of the notebook project in check. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Wed Aug 13 14:22:13 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 13 Aug 2014 14:22:13 -0400 Subject: [IPython-dev] Separating the notebook from the file manager+text editor+terminal In-Reply-To: References: Message-ID: I am simultaneously +1 and -1 :) +1: anything that helps us to write better, more modular code is a huge improvement. As we add the terminal and text editor to the notebook, we will have a great chance to improve these abstractions. I think this will also help us to develop abstractions that enable our different components to be reused in different contexts. I like that and think it is important. Having separate repos for these components *may* even make sense, although I wouldn't start there. -1: I am have always been, and remain, opposed to turning our main notebook app into a general web server project that can load and run arbitrary combination of components. I think our notebook server should run a fixed and consistent set of components for all users. Anything different should be a "new app", not merely the same app reconfigured. Timing: I think the best way to proceed is to start developing the terminal and text editor and see what abstractions start to develop for these N=3 and N=4 cases Dashboard: I too want to push back on the dashboard becoming a full blown file manager. I am also opposed to there being multiple dashboard that we maintain and which our users can choose between. I think this is not a situation where we want to "settle disagreement by making no decision and providing configuration instead." At the same time, we are going to have to support more operations than we currently do. The best model for balancing these concerns that I have seen is the github UI. It basically has our UI, with a few extra things for moving files around. I think we should aim for something like that. On Wed, Aug 13, 2014 at 1:26 PM, MinRK wrote: > I think this sounds like a good plan. I?m not 100% sure how to go about > implementing it, though. It would require that our webapp code be a bit more > modular. We took some steps in that direction with 2.0 - Brian and Zach > moved each component into discrete subpackages and URL areas, and are loaded > with a single call to load_handlers. We could extend this a bit further to > something like load_component, and figure out a way to configure / register > available components. > > -MinRK > > > > On Sun, Aug 10, 2014 at 5:22 PM, Thomas Kluyver wrote: >> >> We've often said that, for people accessing the notebook on a remote >> server, we need to provide certain basic utilities - like the dashboard, >> which is now under pressure to evolve into a file manager, and an in-browser >> terminal and text editor, both of which we intend to add to the notebook. >> >> I propose that we separate development and packaging of the notebook from >> the project to assemble this trio. The new project would live under the >> Jupyter org, and be called something like 'utility-apps'. I see several >> advantages of this: >> >> - A different crowd of developers may be attracted to work on that, >> without having to know anything about the internals of IPython's machinery >> for handling and executing code. For instance, I'm pushing back on turning >> the dashboard into a full blown file manager, because I think that adds a >> lot of extra complexity in a codebase that's already complex, but if it was >> a separate project, it could grow into a file manager without being a >> problem. >> - People running the notebook locally may want to install just the >> notebook component, and not poor web-based clones of applications which they >> have well developed equivalents for already installed. >> - There are lots of potential users of the file manager/text >> editor/terminal trio who probably wouldn't use the notebook: for instance, >> it could be a basic admin interface to any server. >> - It could become a base on which people could write other HTML >> applications to meet their own needs. Following our usual MO, I wouldn't try >> to design it like this straight away, but keep it in mind, and evolve the >> APIs as things mature. >> >> There are still plenty of questions to be answered. How would this new >> component interface with the notebook server? What would the notebook look >> like without the file manager installed? But these could certainly be worked >> out, and I think they're worth working out in order to keep the scope of the >> notebook project in check. >> >> Thomas >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From takowl at gmail.com Wed Aug 13 14:31:24 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 13 Aug 2014 11:31:24 -0700 Subject: [IPython-dev] Separating the notebook from the file manager+text editor+terminal In-Reply-To: References: Message-ID: On 13 August 2014 11:22, Brian Granger wrote: > ...the github UI. It basically has our UI, with a few extra > things for moving files around. > You can move files in the Github UI? I've never come across that. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From damianavila at gmail.com Wed Aug 13 14:43:34 2014 From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=) Date: Wed, 13 Aug 2014 15:43:34 -0300 Subject: [IPython-dev] Separating the notebook from the file manager+text editor+terminal In-Reply-To: References: Message-ID: I also agree on the modularity... with the code base getting bigger and bigger it would be very useful to have the pieces split, and I think that having separate repos is a necessary but not sufficient step towards a real split. I think that having separate repos makes easier to make devs/users to work in specific parts as a team as Thomas pointed out before... Regarding the second Bryan's point, I also thing that our notebook server should run a fixed and consistent set of components for all users. Maybe my vision is a little biased here, but I think the set of notebook, terminal, text-editor and dashboard is the minimal and well balance set of tool to completely achieve the majority of our user's uses cases, despite the variability on the use cases community... About the dashboard, we can (and have) to make it better, and people will continue try to push toward a file manager, because it is the natural evolution... and because probably everyone would like to have something like that in their browsers. In any case, if enough people want to work on that, as a separate repo, we can probably embrace them and replace the dashboard, but it is clear, at least for me and I think fo all, that a full-featured file manager is beyond our scope... 2014-08-13 15:31 GMT-03:00 Thomas Kluyver : > On 13 August 2014 11:22, Brian Granger wrote: > >> ...the github UI. It basically has our UI, with a few extra >> >> things for moving files around. >> > > You can move files in the Github UI? I've never come across that. > > Thomas > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- *Dami?n* -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Wed Aug 13 14:53:28 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 13 Aug 2014 14:53:28 -0400 Subject: [IPython-dev] Separating the notebook from the file manager+text editor+terminal In-Reply-To: References: Message-ID: Yes, it is really cool (moving files in GitHub UI). There are two ways you can change a files directory and both use the same UI: 1) when creating a new file in the UI and 2) when editing a file in the UI. In both cases, the UI shows the name of the file, preceeded by its path. It looks like only the name is editable, but if you type a "/" in the name it turns that into a new subdir in the path and if you press "delete" when in the L-most position of the name, you delete the previous path segment. It is wicked cool and lightweight. On Wed, Aug 13, 2014 at 2:43 PM, Dami?n Avila wrote: > I also agree on the modularity... with the code base getting bigger and > bigger it would be very useful to have the pieces split, and I think that > having separate repos is a necessary but not sufficient step towards a real > split. I think that having separate repos makes easier to make devs/users to > work in specific parts as a team as Thomas pointed out before... > > Regarding the second Bryan's point, I also thing that our notebook server > should run a fixed and consistent set of components for all users. Maybe my > vision is a little biased here, but I think the set of notebook, terminal, > text-editor and dashboard is the minimal and well balance set of tool to > completely achieve the majority of our user's uses cases, despite the > variability on the use cases community... > > About the dashboard, we can (and have) to make it better, and people will > continue try to push toward a file manager, because it is the natural > evolution... and because probably everyone would like to have something like > that in their browsers. In any case, if enough people want to work on that, > as a separate repo, we can probably embrace them and replace the dashboard, > but it is clear, at least for me and I think fo all, that a full-featured > file manager is beyond our scope... > > > 2014-08-13 15:31 GMT-03:00 Thomas Kluyver : >> >> On 13 August 2014 11:22, Brian Granger wrote: >>> >>> ...the github UI. It basically has our UI, with a few extra >>> >>> things for moving files around. >> >> >> You can move files in the Github UI? I've never come across that. >> >> Thomas >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Dami?n > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From tritemio at gmail.com Wed Aug 13 21:56:18 2014 From: tritemio at gmail.com (Antonino Ingargiola) Date: Wed, 13 Aug 2014 18:56:18 -0700 Subject: [IPython-dev] IPython Notebook in read-only mode Message-ID: Hi, I need to share some notebooks with some colleagues from my desktop (windows). Would be possible to configure an ipython notebook server that accepts connection from different machines but serves notebooks in read-only mode? I remember that at some point in the past (maybe around 0.13) I had a configuration like this, but I don't remember if it was just some side effect of the firewall. I could try to install nbviewer but I would also need to setup a second server to host the ipynb files IIUC. Any guidance on how to setup nbviewer for local files on Windows would also help :). Antonio -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Wed Aug 13 22:03:51 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 13 Aug 2014 22:03:51 -0400 Subject: [IPython-dev] IPython Notebook in read-only mode In-Reply-To: References: Message-ID: The multiuser notebook server we are working on will eventually have something like this. But the single user server won't. In the meantime I would use nbviewer, or just nbconvert and a static website. Cheers, Brian On Wed, Aug 13, 2014 at 9:56 PM, Antonino Ingargiola wrote: > Hi, > > I need to share some notebooks with some colleagues from my desktop > (windows). > > Would be possible to configure an ipython notebook server that accepts > connection from different machines but serves notebooks in read-only mode? > > I remember that at some point in the past (maybe around 0.13) I had a > configuration like this, but I don't remember if it was just some side > effect of the firewall. > > I could try to install nbviewer but I would also need to setup a second > server to host the ipynb files IIUC. Any guidance on how to setup nbviewer > for local files on Windows would also help :). > > Antonio > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From zvoros at gmail.com Thu Aug 14 03:04:41 2014 From: zvoros at gmail.com (=?UTF-8?B?Wm9sdMOhbiBWw7Zyw7Zz?=) Date: Thu, 14 Aug 2014 09:04:41 +0200 Subject: [IPython-dev] mathjax equation number is incremented on cntr-enter In-Reply-To: References: <53E45619.4040302@gmail.com> Message-ID: <53EC5F89.1060107@gmail.com> Hi Min, Thanks for the pointer! And now I understand why I didn't find this issue: it has been known for too long a time... Cheers, Zolt?n On 08/13/2014 07:17 PM, MinRK wrote: > > This is a known issue > , where mathjax > generates an equation number for each equation it renders, so they > keep increasing on each render. In order to get proper equation > numbering, all equations (markdown cells) on the page would need to be > re-rendered every time any equation is. There isn?t a good shortcut > for this at the moment, but reloading the page should work. > > -MinRK > > ? > > > On Thu, Aug 7, 2014 at 9:46 PM, Zolt?n V?r?s > wrote: > > Hi all, > > I don't know, whether this is a known issue (a quick search did > not turn up anything), in fact, I don't even know, whether this is > an issue at all, but I believe, the markdown cell did behave > differently in the past. In short, it seems to me that mathjax > increments the equation number, whenever I press Cntr-Enter on a > markdown cell. Here is my notebook, and here is the code that > produces the problem: > > \begin{equation} > x = y > \end{equation} > > I haven't removed anything, yet, the equation is labelled (4). Can > someone comment on this? In particular, whether this was intended? > Is this, perhaps, related to the security problem that was > floating around recently? I am running the latest code from master. > > Cheers, > > Zolt?n > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From takowl at gmail.com Thu Aug 14 14:20:59 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 14 Aug 2014 11:20:59 -0700 Subject: [IPython-dev] Mime type for IPython notebooks Message-ID: In the dev meeting, we discussed what the mime type should be for IPython notebook files. The people in the meeting preferred *application/x-ipynb+json* (we also considered application/x-jupyter-notebook+json, and both of those without the +json suffix). Looking at the official mime type registry [1], +json and +xml are common suffixes to indicate container formats. Does anyone want to argue for a different mime type before we declare application/x-ipynb+json the victor? 1 http://www.iana.org/form/media-types Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug.blank at gmail.com Thu Aug 14 14:55:00 2014 From: doug.blank at gmail.com (Doug Blank) Date: Thu, 14 Aug 2014 14:55:00 -0400 Subject: [IPython-dev] Mime type for IPython notebooks In-Reply-To: References: Message-ID: On Thu, Aug 14, 2014 at 2:20 PM, Thomas Kluyver wrote: > In the dev meeting, we discussed what the mime type should be for IPython > notebook files. The people in the meeting preferred > *application/x-ipynb+json* (we also considered > application/x-jupyter-notebook+json, and both of those without the +json > suffix). Looking at the official mime type registry [1], +json and +xml are > common suffixes to indicate container formats. > > Does anyone want to argue for a different mime type before we declare > application/x-ipynb+json the victor? > Before you wrote this, I would have guessed that the filename extensions on notebooks would probably be changed to something more generic. Thinking that you are going to also attach those 5 letters to the mime-type makes me wonder. So, let me make a couple of points, and try to sway y'all in a more generic naming scheme. An aside, I think you could leave the +json off. The container indication is useful on svg+xml to remind people that it is an image, and also xml. Thus, many browsers can handle it directly. I don't think it makes sense on a general format, unless you are considering adding application/...+xml (or some other container format) at some point, too. Knowing that a format is json doesn't get you much. Jupyter will soon be the name that represents the big picture. Why wouldn't the mime-type also represent the big picture? Also, there isn't any other Jupyter file format, so why not just make it application/x-jupyter ? As for file extension, consider the big picture, too. Maybe: .jnb, .jup, or even .jupyter. Anyway, thanks for asking for arguments; always willing to oblige :) -Doug > > 1 http://www.iana.org/form/media-types > > Thanks, > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Thu Aug 14 15:48:56 2014 From: benjaminrk at gmail.com (MinRK) Date: Thu, 14 Aug 2014 12:48:56 -0700 Subject: [IPython-dev] Mime type for IPython notebooks In-Reply-To: References: Message-ID: On Thu, Aug 14, 2014 at 11:55 AM, Doug Blank wrote: On Thu, Aug 14, 2014 at 2:20 PM, Thomas Kluyver wrote: > >> In the dev meeting, we discussed what the mime type should be for IPython >> notebook files. The people in the meeting preferred >> *application/x-ipynb+json* (we also considered >> application/x-jupyter-notebook+json, and both of those without the +json >> suffix). Looking at the official mime type registry [1], +json and +xml are >> common suffixes to indicate container formats. >> >> Does anyone want to argue for a different mime type before we declare >> application/x-ipynb+json the victor? >> > > Before you wrote this, I would have guessed that the filename extensions > on notebooks would probably be changed to something more generic. Thinking > that you are going to also attach those 5 letters to the mime-type makes me > wonder. So, let me make a couple of points, and try to sway y'all in a more > generic naming scheme. > We wouldn?t have picked .ipynb if we were choosing now, but the cost of changing one letter seems to significantly outweigh the benefits. For a slightly facetious argument: In Latin, there is no distinction between I and J, so technically the name would be IUPYTER, so a .ipynb extension still makes sense for project iupyter :) > An aside, I think you could leave the +json off. The container indication > is useful on svg+xml to remind people that it is an image, and also xml. > Thus, many browsers can handle it directly. I don't think it makes sense on > a general format, unless you are considering adding application/...+xml (or > some other container format) at some point, too. Knowing that a format is > json doesn't get you much. > It doesn?t get you much, but we found a number of examples following the convention on the IANA registry , and it seems helpful to note that it is indeed plain JSON, and the notebook just describes the *structure* of that JSON. > Jupyter will soon be the name that represents the big picture. Why > wouldn't the mime-type also represent the big picture? Also, there isn't > any other Jupyter file format, so why not just make it > application/x-jupyter ? As for file extension, consider the big picture, > too. Maybe: .jnb, .jup, or even .jupyter. > I think we would keep notebook as part of it, so honestly the most likely candidate is .jpynb, which is almost identical to the existing .ipynb. There could easily be other non-notebook Jupyter-related files in the future. > Anyway, thanks for asking for arguments; always willing to oblige :) > Thanks! -MinRK > -Doug > > >> >> 1 http://www.iana.org/form/media-types >> >> Thanks, >> Thomas >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkpaustin at gmail.com Thu Aug 14 19:55:37 2014 From: mkpaustin at gmail.com (Phil Austin) Date: Thu, 14 Aug 2014 23:55:37 +0000 (UTC) Subject: [IPython-dev] making emacs keybindings the ipython default Message-ID: I need a hint on how to turn on codemirror's emacs keybindings in custom.js for IPython 2.1. I've found https://gist.github.com/minrk/5940801 and https://nbviewer.jupyter.org/github/ivanov/scipy2014/blob/master/ v%20in%20IPython.ipynb#ipython-vimception but when I insert %%javascript // change the mode of all current and future CodeMirror instances function to(mode) { var mode = mode || 'vim' // first let's apply vim mode to all current cells function to_mode(c) { return c.code_mirror.setOption('keyMap', mode);}; IPython.notebook.get_cells().map(to_mode); // apply the mode to future cells created IPython.Cell.options_default.cm_config.keyMap = mode; } require(["/static/components/codemirror/keymap/emacs"], function (emacs) { to('emacs'); console.log('emacs.js loaded'); }); into ~/.ipython/profile_default/static/custom/custom.js the console.log (OSX Mavericks, Chrome) reports a 404: 2014-08-14 16:43:08.685 [NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). WARNING:tornado.access:404 GET /static/components/codemirror/keymap/emacs (::1) 0.88ms referer=http://localhost:8889/notebooks/timetable.ipynbn 2014-08-14 16:43:10.988 [NotebookApp] Kernel started: 16eaa587-e1c8-4f51-8951- 3d6e60ab3ad6 and how to actually call the callback from custom.js seems to have gotten deleted from the bottom of the notebook? thanks, Phil From pi at berkeley.edu Thu Aug 14 20:23:37 2014 From: pi at berkeley.edu (Paul Ivanov) Date: Thu, 14 Aug 2014 17:23:37 -0700 Subject: [IPython-dev] making emacs keybindings the ipython default In-Reply-To: References: Message-ID: <20140815002337.GG3924@HbI-OTOH.berkeley.edu> Hi Phil, Sorry about the goof, you need to either put ".js" after emacs the way I had it before, or not using a leading forward slash, and just do the example below (adjusted for how you should add it to custom.js) $([IPython.events]).on('app_initialized.NotebookApp', function(){ function to(mode) { var mode = mode || 'vim' // first let's apply vim mode to all current cells function to_mode(c) { return c.code_mirror.setOption('keyMap', mode);}; IPython.notebook.get_cells().map(to_mode); // apply the mode to future cells created IPython.Cell.options_default.cm_config.keyMap = mode; } require(["components/codemirror/keymap/emacs"], function (emacs) { to('emacs'); console.log('emacs.js loaded'); }); }) Thanks for the bug report, let me go fix that in the notebook you referenced, -- _ / \ A* \^ - ,./ _.`\\ / \ / ,--.S \/ \ / `"~,_ \ \ __o ? _ \<,_ /:\ --(_)/-(_)----.../ | \ --------------.......J Paul Ivanov ipython and matplotlib core developer @ivanov on github and twitter http://pirsquared.org From mkpaustin at gmail.com Thu Aug 14 21:55:02 2014 From: mkpaustin at gmail.com (taleb brost) Date: Thu, 14 Aug 2014 18:55:02 -0700 Subject: [IPython-dev] making emacs keybindings the ipython default In-Reply-To: <20140815002337.GG3924@HbI-OTOH.berkeley.edu> References: <20140815002337.GG3924@HbI-OTOH.berkeley.edu> Message-ID: On Thu, Aug 14, 2014 at 5:23 PM, Paul Ivanov wrote: > Hi Phil, > > Sorry about the goof, you need to either put ".js" after emacs > the way I had it before, or not using a leading forward slash, > and just do the example below (adjusted for how you should add it > to custom.js) > thanks, for the quick response -- almost there, in that, the 404 error is silenced, but when I start a new notebook I'm not getting the console message 'emacs.js loaded' and ctrl-p etc. are no-ops. Is there a final step to call the function loaded by require aside from initializing the notebook? My javascript knowledge is just above zero. best, Phil -------------- next part -------------- An HTML attachment was scrubbed... URL: From claresloggett at gmail.com Fri Aug 15 10:48:56 2014 From: claresloggett at gmail.com (Clare Sloggett) Date: Sat, 16 Aug 2014 00:48:56 +1000 Subject: [IPython-dev] IPython Notebook in read-only mode In-Reply-To: References: Message-ID: Hi Antonio, If you're in an office with people and you don't want to set up an actual website, you could also just send them the html file, or put it in a file sharing space in your office, or Dropbox. Even if your colleagues aren't particularly techie, they only need to "open" the html file to display it a browser. But, a read-only mode for Notebook is something I've wanted too. The reason is that IPython Notebook provides its own authentication mechanism, so something can be at a public address, but still relatively private. And it's trivial to get going, since Notebook runs a webserver for you. Of course I can use another authentication system on the static html, like Dropbox sharing, or set up a webserver with a password-protected area, but it would be pretty nice if Notebook supported read-only access as fully as it supports read/write access. Cheers, Clare On 14 August 2014 12:03, Brian Granger wrote: > The multiuser notebook server we are working on will eventually have > something like this. But the single user server won't. In the meantime > I would use nbviewer, or just nbconvert and a static website. > > Cheers, > > Brian > > On Wed, Aug 13, 2014 at 9:56 PM, Antonino Ingargiola > wrote: > > Hi, > > > > I need to share some notebooks with some colleagues from my desktop > > (windows). > > > > Would be possible to configure an ipython notebook server that accepts > > connection from different machines but serves notebooks in read-only > mode? > > > > I remember that at some point in the past (maybe around 0.13) I had a > > configuration like this, but I don't remember if it was just some side > > effect of the firewall. > > > > I could try to install nbviewer but I would also need to setup a second > > server to host the ipynb files IIUC. Any guidance on how to setup > nbviewer > > for local files on Windows would also help :). > > > > Antonio > > > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Fri Aug 15 11:01:08 2014 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Fri, 15 Aug 2014 16:01:08 +0100 Subject: [IPython-dev] IPython Notebook in read-only mode In-Reply-To: References: Message-ID: <409FED76-DF37-4375-895A-A10FFE5E2C4F@gmail.com> Nbviewer has an (unsupported) option at startup to serve ipynb from cwd. I suppose that will be enough for you. Envoy? de mon iPhone > Le 14 ao?t 2014 ? 02:56, Antonino Ingargiola a ?crit : > > Hi, > > I need to share some notebooks with some colleagues from my desktop (windows). > > Would be possible to configure an ipython notebook server that accepts connection from different machines but serves notebooks in read-only mode? > > I remember that at some point in the past (maybe around 0.13) I had a configuration like this, but I don't remember if it was just some side effect of the firewall. > > I could try to install nbviewer but I would also need to setup a second server to host the ipynb files IIUC. Any guidance on how to setup nbviewer for local files on Windows would also help :). > > Antonio > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From mankoff at gmail.com Fri Aug 15 13:49:03 2014 From: mankoff at gmail.com (Ken Mankoff) Date: Fri, 15 Aug 2014 13:49:03 -0400 Subject: [IPython-dev] making emacs keybindings the ipython default In-Reply-To: References: <20140815002337.GG3924@HbI-OTOH.berkeley.edu> Message-ID: Just a note that if you want emacs + IPython notebook, there is a project for working with notebook files in emacs. See the EIN https://tkf.github.io/emacs-ipython-notebook/ -k. On Thu, Aug 14, 2014 at 9:55 PM, taleb brost wrote: > On Thu, Aug 14, 2014 at 5:23 PM, Paul Ivanov wrote: > >> Hi Phil, >> >> Sorry about the goof, you need to either put ".js" after emacs >> the way I had it before, or not using a leading forward slash, >> and just do the example below (adjusted for how you should add it >> to custom.js) >> > > > thanks, for the quick response -- almost there, in that, the 404 error > is silenced, but > when I start a new notebook I'm not getting the console message 'emacs.js > loaded' > and ctrl-p etc. are no-ops. Is there a final step to call the function > loaded by require aside > from initializing the notebook? > My javascript knowledge is just above zero. best, Phil > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From efiring at hawaii.edu Fri Aug 15 16:22:12 2014 From: efiring at hawaii.edu (Eric Firing) Date: Fri, 15 Aug 2014 10:22:12 -1000 Subject: [IPython-dev] ipython slowdown with qt In-Reply-To: <574A53C0-860B-4622-B23A-E14E8693404B@astro.physik.uni-goettingen.de> References: <53ED08DF.90304@gmail.com> <574A53C0-860B-4622-B23A-E14E8693404B@astro.physik.uni-goettingen.de> Message-ID: <53EE6BF4.2050407@hawaii.edu> On 2014/08/15, 9:37 AM, Derek Homeier wrote: > When using MPL with ipython ?pylab and the Quartz version of PyQT4, > the interpreter seems to be slow down extremely after running for a > little while. Weirdly this is not connected to any graphics display > and in fact happens even without any plotting window opened, i.e. the > ipython shell just randomly becomes completely unresponsive and hangs > for several seconds on simple tasks like typing or navigating through > history. The plotting itself actually does not appear to perform any > worse than it used to under Mountain Lion. [I'm switching the subject because my comments below relate to ipython and matplotlib, and are no longer Anaconda-specific.] Derek, Thanks. A few days ago, when I switched from testing on linux to testing on osx, exactly this ipython slowdown was happening to me--but I lost track of what combination of versions and invocations was causing it. Therefore I have been concentrating on the severe problem which was, for me, 100% repeatable, and involved macosx backend, not Qt. I expect the macosx-relatec problem will go away after Ilan uploads the revised Anaconda ipython for python 3. Now I find I can repeat the ipython problem on Homebrew python 3 (framework--with Quartz app) and Anaconda with the un-fixed ipython (which is running without starting a Quartz app): ipython --pylab=qt Leave it alone for a bit. Try scrolling through history. Long delay, even in responding to Ctrl-C. Evidently key events are stacking up and not being processed. Now try: ipython %pylab qt I see the slowdown with this, also. The response delay seems to get worse with time. It renders the session unusable after only a few minutes. ipython %gui qt And I still see it, so this appears to be a problem in ipython's PyQt4 gui handling, not directly related to matplotlib. All on Mavericks, running ipython from Apple's terminal. Eric From jenshnielsen at gmail.com Fri Aug 15 17:05:14 2014 From: jenshnielsen at gmail.com (Jens Nielsen) Date: Fri, 15 Aug 2014 22:05:14 +0100 Subject: [IPython-dev] ipython slowdown with qt In-Reply-To: <53EE6BF4.2050407@hawaii.edu> References: <53ED08DF.90304@gmail.com> <574A53C0-860B-4622-B23A-E14E8693404B@astro.physik.uni-goettingen.de> <53EE6BF4.2050407@hawaii.edu> Message-ID: While I can reproduce the issue using %gui qt I can also reproduce it with the WX backend (%qui wx) with more or less the same symptoms. However, I don't see the issue with either of the 'tk' or the 'osx' backends. And yes the issue is reproducible in a python installation without any mpl installed. /Jens On Fri, Aug 15, 2014 at 9:22 PM, Eric Firing wrote: > On 2014/08/15, 9:37 AM, Derek Homeier wrote: > > When using MPL with ipython ?pylab and the Quartz version of PyQT4, > > the interpreter seems to be slow down extremely after running for a > > little while. Weirdly this is not connected to any graphics display > > and in fact happens even without any plotting window opened, i.e. the > > ipython shell just randomly becomes completely unresponsive and hangs > > for several seconds on simple tasks like typing or navigating through > > history. The plotting itself actually does not appear to perform any > > worse than it used to under Mountain Lion. > > [I'm switching the subject because my comments below relate to ipython > and matplotlib, and are no longer Anaconda-specific.] > > Derek, > > Thanks. A few days ago, when I switched from testing on linux to > testing on osx, exactly this ipython slowdown was happening to me--but I > lost track of what combination of versions and invocations was causing > it. Therefore I have been concentrating on the severe problem which > was, for me, 100% repeatable, and involved macosx backend, not Qt. I > expect the macosx-relatec problem will go away after Ilan uploads the > revised Anaconda ipython for python 3. > > Now I find I can repeat the ipython problem on Homebrew python 3 > (framework--with Quartz app) and Anaconda with the un-fixed ipython > (which is running without starting a Quartz app): > > ipython --pylab=qt > > Leave it alone for a bit. Try scrolling through history. Long delay, > even in responding to Ctrl-C. Evidently key events are stacking up and > not being processed. Now try: > > ipython > %pylab qt > > I see the slowdown with this, also. The response delay seems to get > worse with time. It renders the session unusable after only a few minutes. > > ipython > %gui qt > > And I still see it, so this appears to be a problem in ipython's PyQt4 > gui handling, not directly related to matplotlib. All on Mavericks, > running ipython from Apple's terminal. > > Eric > > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From efiring at hawaii.edu Fri Aug 15 17:55:33 2014 From: efiring at hawaii.edu (Eric Firing) Date: Fri, 15 Aug 2014 11:55:33 -1000 Subject: [IPython-dev] ipython slowdown with qt In-Reply-To: References: <53ED08DF.90304@gmail.com> <574A53C0-860B-4622-B23A-E14E8693404B@astro.physik.uni-goettingen.de> <53EE6BF4.2050407@hawaii.edu> Message-ID: <53EE81D5.8010309@hawaii.edu> On 2014/08/15, 11:05 AM, Jens Nielsen wrote: > While I can reproduce the issue using %gui qt I can also reproduce it > with the WX backend (%qui wx) with more or less the same symptoms. > However, I don't see the issue with either of the 'tk' or the 'osx' > backends. And yes the issue is reproducible in a python installation > without any mpl installed. > > /Jens > This is https://github.com/ipython/ipython/issues/6189. Eric From hughesadam87 at gmail.com Sun Aug 17 13:54:59 2014 From: hughesadam87 at gmail.com (hugadams) Date: Sun, 17 Aug 2014 10:54:59 -0700 (PDT) Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this Message-ID: <1408298099961-5068197.post@n6.nabble.com> (Please disregard if doublepost; having some subscription issues) Hello, We are developing a spectroscopy library and want to put in some interactive plots/utilities to make the software a bit more approachable to novices. I'd really like to build the entire interactive plotting interface through the notebook's new widget system. My reasoning is as follows: - All static plots are already matplotlib-based, so we wouldn't have to write a seperate plotting api - Notebooks are inherently browser-ready, so sharing plots would be great - Is it likely that static nbviewer will eventually support widgets? Since the data is required, I figured not... - Our tutorials are already in notebooks, so we'd like to be consistent. I've watched all the videos I can find from the widgets, and think I'm fine with the Python end; however, am finding it hard to come by lots of information on controlling the layout of the widgets. I know it's mostly done in javascript, and I think the most complex view we will have will look something like this: https://sites.google.com/site/shigemorita/home/2dshige How much time would you estimate a javascript novice ought to spend on getting something like that in the notebook? Can anyone share tutorials for the layout portion of the widget API? Any information is helpful. Lastly, does anyone have any hacky ideas of how to bury the source code so that the widgets merely display, rather than the user copy/pasting the widget source into a cell? Thanks -- View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197.html Sent from the IPython - Development mailing list archive at Nabble.com. From illanos at cal.berkeley.edu Tue Aug 19 15:02:45 2014 From: illanos at cal.berkeley.edu (Imelda Llanos de Luna) Date: Tue, 19 Aug 2014 12:02:45 -0700 Subject: [IPython-dev] IPython-Stakeholder Analysis Message-ID: Thanks for the warm welcome last week, everyone. If it's okay, while the DRAFT of the Business Need is under review, I'd like to proceed with performing a Stakeholder Analysis of IPython project to show the different stakeholder groups and layers. I haven't spotted one on the various links, although different pages provide a good idea of most of the stakeholder groups if not all. If there is a list or visualization and I missed seeing it, let me know. Otherwise, I can draft something. Also, if you'd like an example of a Stakeholder Analysis, please contact me at illanos(at)cal.berkeley .edu , and I'll email you a link to one I completed at UC Berkeley. As for the draft of the Business Need, it can be further elaborated on, e.g. more benchmarking of competitors if it doesn't exist and it would be helpful. I would go into more detail of who they are, their size, their product, their customers, their reach, their cost/fees, etc. Just let me know with any feedback when you have a moment to provide feedback. Thanks a bunch. Sincerely,Imelda Llanos de Luna -------------- next part -------------- An HTML attachment was scrubbed... URL: From cyrille.rossant at gmail.com Wed Aug 20 04:15:38 2014 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Wed, 20 Aug 2014 10:15:38 +0200 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: <1408298099961-5068197.post@n6.nabble.com> References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: Hi Adam, I'll do a tutorial about IPython widgets at EuroSciPy in a week [1]. I'm notably planning to demonstrate how to build, in the notebook, a GUI like the one you showed. I'll put here the link to the GitHub repo when everything's ready. Best, Cyrille [1] https://www.euroscipy.org/2014/schedule/presentation/60/ 2014-08-17 19:54 GMT+02:00 hugadams : > (Please disregard if doublepost; having some subscription issues) > > Hello, > > We are developing a spectroscopy library and want to put in some interactive > plots/utilities to make the software a bit more approachable to novices. > I'd really like to build the entire interactive plotting interface through > the notebook's new widget system. My reasoning is as follows: > > - All static plots are already matplotlib-based, so we wouldn't have to > write a seperate plotting api > - Notebooks are inherently browser-ready, so sharing plots would be great > - Is it likely that static nbviewer will eventually support > widgets? Since the data is required, I figured not... > - Our tutorials are already in notebooks, so we'd like to be consistent. > > I've watched all the videos I can find from the widgets, and think I'm fine > with the Python end; however, am finding it hard to come by lots of > information on controlling the layout of the widgets. I know it's mostly > done in javascript, and I think the most complex view we will have will look > something like this: > > https://sites.google.com/site/shigemorita/home/2dshige > > How much time would you estimate a javascript novice ought to spend on > getting something like that in the notebook? Can anyone share tutorials for > the layout portion of the widget API? Any information is helpful. Lastly, > does anyone have any hacky ideas of how to bury the source code so that the > widgets merely display, rather than the user copy/pasting the widget source > into a cell? > > Thanks From rgbkrk at gmail.com Mon Aug 18 01:09:12 2014 From: rgbkrk at gmail.com (Kyle Kelley) Date: Mon, 18 Aug 2014 00:09:12 -0500 Subject: [IPython-dev] Url route to download a notebook and open it In-Reply-To: References: Message-ID: Going after the security issue, the nice thing is that we're now signing the notebooks. No javascript from a notebook would be run. iframe embedding is now blocked by the X-Frame-Options header as well. With the notebook prior to v2, as an attacker I would create a site that opens an iframe to a download route (or provides a link) to a malicious notebook (hosted on one of the whitelisted domains, like githubusercontent.com). That's not to say that there wouldn't be a hole elsewhere with a download route. The thing about making an explicit action on the user's part is to make this avenue for attack less easy. On Thu, Aug 7, 2014 at 10:24 AM, Paddy Mullen wrote: > It would be useful to have an url route that downloaded a notebook and > opened it. > > I could see a route like http://localhost:8888/download/ > https://raw.githubusercontent.com/ipython/ipython/master/examples/Notebook/SymPy.ipynb > > that would download the SymPy notebook and open it. > > This would make it easy to link to generated notebooks. The alternative > workflow now is to provide a download link, and have the user drag that > file into the notebook filebrowser. > > I realize that this would be a security risk, so it would probably be a > feature that was best disabled by default. A configurable whitelist of > allowable download domains could help a lot. > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Kyle Kelley (@rgbkrk ; http://lambdaops.com) -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug.blank at gmail.com Wed Aug 20 07:22:00 2014 From: doug.blank at gmail.com (Doug Blank) Date: Wed, 20 Aug 2014 07:22:00 -0400 Subject: [IPython-dev] IPython3/Jupyter Magics for Python-based kernels Message-ID: IPython developers, One of the nice things that the IPython team did for the next version was to create a base class for creating new kernels for code written in Python, called wrapper kernels [1]. Unfortunately, such kernels don't have ! shell commands, nor magics. This is understandable, as those are tightly integrated with IPython's Python. However, one could get some of the basic functionality of these features on top of the wrapper kernel, in a generic way. For example, you could run shell commands, or have some magics that operate independent of the kernel. We have created a prototype of magics for wrapper kernels [2]. The code is in magickernel.py, and the magics are in the folder magics. The current prototype works like this: ``` In [1]: %magics Line magics: %cd PATH - change current directory of session %connect_info - show connection information %download URL [FILENAME] - download file from URL %html CODE - display code as HTML %install_magic URL - download and install magic from URL %javascript CODE - send code as JavaScript %latex TEXT - display text as LaTex %magics - show installed magics %reload_magics - reload the magics from the installed files %shell COMMAND - run the line as a shell command %time COMMAND - show time to run line Cell magics: %%file FILENAME - write contents of cell to file %%html - display contents of cell as HTML %%javascript - send contents of cell as JavaScript %%latex - display contents of cell as LaTeX %%shell - run the contents of the cell as shell commands %%time - show time to run cell Shell shortcut: ! COMMAND ... - execute command in shell Any cell magic can be made persistent for rest of session by using %%% prefix. ``` The idea is to be able to easily add some standard-like magics to 3rd-party kernels by using MagicKernel. See the sample wrapperkernel EchoKernel with magics [3]. Also, the idea here is not to replicate too much what comes with IPython's Python, but to provide a small, simple core set of magics that can't be easily done in the kernel language. You can install MagicKernel from source, or using: $ pip install calico --upgrade This is a prototype (with some bugs), and we very much hope for others to provide feedback or code, and more work needs to be done. What magics would you need in your 3rd-party kernel, that can be accomplished this way? Will this design be sufficient? We hope that this is useful! -Doug [1] - http://ipython.org/ipython-doc/dev/development/wrapperkernels.html [2] - https://bitbucket.org/ipre/calico/src/master/src/calico/ [3] - https://bitbucket.org/ipre/calico/src/master/src/echo_kernel/echo_kernel.py -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Wed Aug 20 11:21:26 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 20 Aug 2014 11:21:26 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: <1408298099961-5068197.post@n6.nabble.com> References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: * You should be able to do this without any JavaScript - just python. * I would wait a few weeks - we have some new container logic that is about to be merged into ipython master that will make this much easier. * After those changes are in I would estimate it will take about 1-2 days to create this UI once you learn the widget APIs. May take more time if you need to wire up complex event handling for the UI elements. Cheers, Brian On Sun, Aug 17, 2014 at 1:54 PM, hugadams wrote: > (Please disregard if doublepost; having some subscription issues) > > Hello, > > We are developing a spectroscopy library and want to put in some interactive > plots/utilities to make the software a bit more approachable to novices. > I'd really like to build the entire interactive plotting interface through > the notebook's new widget system. My reasoning is as follows: > > - All static plots are already matplotlib-based, so we wouldn't have to > write a seperate plotting api > - Notebooks are inherently browser-ready, so sharing plots would be great > - Is it likely that static nbviewer will eventually support > widgets? Since the data is required, I figured not... > - Our tutorials are already in notebooks, so we'd like to be consistent. > > I've watched all the videos I can find from the widgets, and think I'm fine > with the Python end; however, am finding it hard to come by lots of > information on controlling the layout of the widgets. I know it's mostly > done in javascript, and I think the most complex view we will have will look > something like this: > > https://sites.google.com/site/shigemorita/home/2dshige > > How much time would you estimate a javascript novice ought to spend on > getting something like that in the notebook? Can anyone share tutorials for > the layout portion of the widget API? Any information is helpful. Lastly, > does anyone have any hacky ideas of how to bury the source code so that the > widgets merely display, rather than the user copy/pasting the widget source > into a cell? > > Thanks > > > > -- > View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197.html > Sent from the IPython - Development mailing list archive at Nabble.com. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From ellisonbg at gmail.com Wed Aug 20 11:22:50 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 20 Aug 2014 11:22:50 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: Oh a few more things: * Plotting is supported * nbviewer support for widgets is currently being worked on * To hide the code, I would write the code in a spearate .py file, and import a top level function and call it. On Wed, Aug 20, 2014 at 11:21 AM, Brian Granger wrote: > * You should be able to do this without any JavaScript - just python. > * I would wait a few weeks - we have some new container logic that is > about to be merged into ipython master that will make this much > easier. > * After those changes are in I would estimate it will take about 1-2 > days to create this UI once you learn the widget APIs. May take more > time if you need to wire up complex event handling for the UI > elements. > > Cheers, > > Brian > > On Sun, Aug 17, 2014 at 1:54 PM, hugadams wrote: >> (Please disregard if doublepost; having some subscription issues) >> >> Hello, >> >> We are developing a spectroscopy library and want to put in some interactive >> plots/utilities to make the software a bit more approachable to novices. >> I'd really like to build the entire interactive plotting interface through >> the notebook's new widget system. My reasoning is as follows: >> >> - All static plots are already matplotlib-based, so we wouldn't have to >> write a seperate plotting api >> - Notebooks are inherently browser-ready, so sharing plots would be great >> - Is it likely that static nbviewer will eventually support >> widgets? Since the data is required, I figured not... >> - Our tutorials are already in notebooks, so we'd like to be consistent. >> >> I've watched all the videos I can find from the widgets, and think I'm fine >> with the Python end; however, am finding it hard to come by lots of >> information on controlling the layout of the widgets. I know it's mostly >> done in javascript, and I think the most complex view we will have will look >> something like this: >> >> https://sites.google.com/site/shigemorita/home/2dshige >> >> How much time would you estimate a javascript novice ought to spend on >> getting something like that in the notebook? Can anyone share tutorials for >> the layout portion of the widget API? Any information is helpful. Lastly, >> does anyone have any hacky ideas of how to bury the source code so that the >> widgets merely display, rather than the user copy/pasting the widget source >> into a cell? >> >> Thanks >> >> >> >> -- >> View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197.html >> Sent from the IPython - Development mailing list archive at Nabble.com. >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > @ellisonbg on Twitter and GitHub > bgranger at calpoly.edu and ellisonbg at gmail.com -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From nick.bollweg at gmail.com Wed Aug 20 18:15:27 2014 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Wed, 20 Aug 2014 18:15:27 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: Didn't see this when it was posted (and then didn't see cyrille's and brian's posts until just now! curse you gmail!), but in the meantime, hacked this together: - http://nbviewer.ipython.org/gist/anonymous/840b5cec9e19f3e39090 Screenshot: - http://i.imgur.com/wP3VmWK.png Obviously lots of room for improvement, but I think you can see you can be very successful without writing any javacript at all. There are some tricks: add_class and remove_class only happen to something that is actually on the page. As for wrapping it all: you'll end up exposing: - a function that returns the dialog widget - a function something that handles the add_remove class cleanup - it might be fixed in the new stuff - a function that will do both of the above and actually call IPython.display.display - this should also return the widget, in case people want custom handlers, I guess? As Brian says, though: really some great stuff coming up! On Wed, Aug 20, 2014 at 4:15 AM, Cyrille Rossant wrote: > Hi Adam, > > I'll do a tutorial about IPython widgets at EuroSciPy in a week [1]. > I'm notably planning to demonstrate how to build, in the notebook, a > GUI like the one you showed. I'll put here the link to the GitHub repo > when everything's ready. > > Best, > Cyrille > > [1] https://www.euroscipy.org/2014/schedule/presentation/60/ > > 2014-08-17 19:54 GMT+02:00 hugadams : > > (Please disregard if doublepost; having some subscription issues) > > > > Hello, > > > > We are developing a spectroscopy library and want to put in some > interactive > > plots/utilities to make the software a bit more approachable to novices. > > I'd really like to build the entire interactive plotting interface > through > > the notebook's new widget system. My reasoning is as follows: > > > > - All static plots are already matplotlib-based, so we wouldn't have > to > > write a seperate plotting api > > - Notebooks are inherently browser-ready, so sharing plots would be > great > > - Is it likely that static nbviewer will eventually support > > widgets? Since the data is required, I figured not... > > - Our tutorials are already in notebooks, so we'd like to be > consistent. > > > > I've watched all the videos I can find from the widgets, and think I'm > fine > > with the Python end; however, am finding it hard to come by lots of > > information on controlling the layout of the widgets. I know it's mostly > > done in javascript, and I think the most complex view we will have will > look > > something like this: > > > > https://sites.google.com/site/shigemorita/home/2dshige > > > > How much time would you estimate a javascript novice ought to spend on > > getting something like that in the notebook? Can anyone share tutorials > for > > the layout portion of the widget API? Any information is helpful. > Lastly, > > does anyone have any hacky ideas of how to bury the source code so that > the > > widgets merely display, rather than the user copy/pasting the widget > source > > into a cell? > > > > Thanks > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hughesadam87 at gmail.com Wed Aug 20 18:29:26 2014 From: hughesadam87 at gmail.com (Adam Hughes) Date: Wed, 20 Aug 2014 18:29:26 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: Guys, Thanks for the help! Nicholas, that's amazing; thank you for hacking this just for our sake! That's really cool of you. The mailing list was having issues the last few days, so I actually started a github discussion on this thread. I'd just like to share that, where Doug graciously helped me a great deal, and I posted a few followup concerns. Obviously they may become moot with all of the cool stuff coming up. Here's the link though in case you'd like to check it out and add anything further. As I elaborate on that link, the holy grail for us would be able to load widgets anytime the user wants a GUI. These widgets would need to accept input data, and return output data. For example in the example Nicholas shared, I'd like the user to be able to pass in a matrix, calculate the synchronous spectra, and then use that for further analysis in subsequent cells in the primary notebook. Having that as a capability would be huge for us, because we envision a mixed API/GUI workflow. https://github.com/ipython/ipython/issues/6336 On Wed, Aug 20, 2014 at 6:15 PM, Nicholas Bollweg wrote: > Didn't see this when it was posted (and then didn't see cyrille's and > brian's posts until just now! curse you gmail!), but in the meantime, > hacked this together: > > > - http://nbviewer.ipython.org/gist/anonymous/840b5cec9e19f3e39090 > > Screenshot: > > - http://i.imgur.com/wP3VmWK.png > > Obviously lots of room for improvement, but I think you can see you can be > very successful without writing any javacript at all. > > There are some tricks: add_class and remove_class only happen to something > that is actually on the page. > > As for wrapping it all: you'll end up exposing: > > - a function that returns the dialog widget > - a function something that handles the add_remove class cleanup > - it might be fixed in the new stuff > - a function that will do both of the above and actually call > IPython.display.display > - this should also return the widget, in case people want custom > handlers, I guess? > > As Brian says, though: really some great stuff coming up! > > > On Wed, Aug 20, 2014 at 4:15 AM, Cyrille Rossant < > cyrille.rossant at gmail.com> wrote: > >> Hi Adam, >> >> I'll do a tutorial about IPython widgets at EuroSciPy in a week [1]. >> I'm notably planning to demonstrate how to build, in the notebook, a >> GUI like the one you showed. I'll put here the link to the GitHub repo >> when everything's ready. >> >> Best, >> Cyrille >> >> [1] https://www.euroscipy.org/2014/schedule/presentation/60/ >> >> 2014-08-17 19:54 GMT+02:00 hugadams : >> > (Please disregard if doublepost; having some subscription issues) >> > >> > Hello, >> > >> > We are developing a spectroscopy library and want to put in some >> interactive >> > plots/utilities to make the software a bit more approachable to novices. >> > I'd really like to build the entire interactive plotting interface >> through >> > the notebook's new widget system. My reasoning is as follows: >> > >> > - All static plots are already matplotlib-based, so we wouldn't have >> to >> > write a seperate plotting api >> > - Notebooks are inherently browser-ready, so sharing plots would be >> great >> > - Is it likely that static nbviewer will eventually support >> > widgets? Since the data is required, I figured not... >> > - Our tutorials are already in notebooks, so we'd like to be >> consistent. >> > >> > I've watched all the videos I can find from the widgets, and think I'm >> fine >> > with the Python end; however, am finding it hard to come by lots of >> > information on controlling the layout of the widgets. I know it's >> mostly >> > done in javascript, and I think the most complex view we will have will >> look >> > something like this: >> > >> > https://sites.google.com/site/shigemorita/home/2dshige >> > >> > How much time would you estimate a javascript novice ought to spend on >> > getting something like that in the notebook? Can anyone share >> tutorials for >> > the layout portion of the widget API? Any information is helpful. >> Lastly, >> > does anyone have any hacky ideas of how to bury the source code so that >> the >> > widgets merely display, rather than the user copy/pasting the widget >> source >> > into a cell? >> > >> > Thanks >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Aug 20 19:07:05 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 20 Aug 2014 16:07:05 -0700 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: On Wed, Aug 20, 2014 at 3:29 PM, Adam Hughes wrote: > These widgets would need to accept input data, and return output data. > For example in the example Nicholas shared, I'd like the user to be able > to pass in a matrix, calculate the synchronous spectra, and then use that > for further analysis in subsequent cells in the primary notebook. Having > that as a capability would be huge for us, because we envision a mixed > API/GUI workflow. One option here would be to provide a text field for the user to name the variables they want returned for any given output (or you can fix those if you want), and then have your kernel-side code store them in the user's namespace directly. Here's how you can immediately write variables in ipython: In [7]: ip = get_ipython() In [8]: a = 'hello' In [9]: ip.user_ns['a'] Out[9]: 'hello' In [10]: ip.user_ns['a'] = 100 In [11]: a Out[11]: 100 We can think further about whether other paradigms might be cleaner, but I'm not sure that making variables be the return value of the widget itself will work all that well. But in the meantime, and until we decide what the best solution is in the long term, this will already work and it uses a 100% stable IPython api. It may be in the end this *is* a long-term solution we all like, but even if not, I think it's pretty serviceable. Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Wed Aug 20 19:25:48 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 20 Aug 2014 19:25:48 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: The need to do the messy add_class stuff after they are on the page will go away. We have new HBox and VBox classes that automatially do the layouts. That is the "great new stuff". Glad you got this far! On Wed, Aug 20, 2014 at 6:15 PM, Nicholas Bollweg wrote: > Didn't see this when it was posted (and then didn't see cyrille's and > brian's posts until just now! curse you gmail!), but in the meantime, hacked > this together: > > http://nbviewer.ipython.org/gist/anonymous/840b5cec9e19f3e39090 > > Screenshot: > > http://i.imgur.com/wP3VmWK.png > > Obviously lots of room for improvement, but I think you can see you can be > very successful without writing any javacript at all. > > There are some tricks: add_class and remove_class only happen to something > that is actually on the page. > > As for wrapping it all: you'll end up exposing: > > a function that returns the dialog widget > a function something that handles the add_remove class cleanup > > it might be fixed in the new stuff > > a function that will do both of the above and actually call > IPython.display.display > > this should also return the widget, in case people want custom handlers, I > guess? > > As Brian says, though: really some great stuff coming up! > > > On Wed, Aug 20, 2014 at 4:15 AM, Cyrille Rossant > wrote: >> >> Hi Adam, >> >> I'll do a tutorial about IPython widgets at EuroSciPy in a week [1]. >> I'm notably planning to demonstrate how to build, in the notebook, a >> GUI like the one you showed. I'll put here the link to the GitHub repo >> when everything's ready. >> >> Best, >> Cyrille >> >> [1] https://www.euroscipy.org/2014/schedule/presentation/60/ >> >> 2014-08-17 19:54 GMT+02:00 hugadams : >> > (Please disregard if doublepost; having some subscription issues) >> > >> > Hello, >> > >> > We are developing a spectroscopy library and want to put in some >> > interactive >> > plots/utilities to make the software a bit more approachable to novices. >> > I'd really like to build the entire interactive plotting interface >> > through >> > the notebook's new widget system. My reasoning is as follows: >> > >> > - All static plots are already matplotlib-based, so we wouldn't have >> > to >> > write a seperate plotting api >> > - Notebooks are inherently browser-ready, so sharing plots would be >> > great >> > - Is it likely that static nbviewer will eventually support >> > widgets? Since the data is required, I figured not... >> > - Our tutorials are already in notebooks, so we'd like to be >> > consistent. >> > >> > I've watched all the videos I can find from the widgets, and think I'm >> > fine >> > with the Python end; however, am finding it hard to come by lots of >> > information on controlling the layout of the widgets. I know it's >> > mostly >> > done in javascript, and I think the most complex view we will have will >> > look >> > something like this: >> > >> > https://sites.google.com/site/shigemorita/home/2dshige >> > >> > How much time would you estimate a javascript novice ought to spend on >> > getting something like that in the notebook? Can anyone share tutorials >> > for >> > the layout portion of the widget API? Any information is helpful. >> > Lastly, >> > does anyone have any hacky ideas of how to bury the source code so that >> > the >> > widgets merely display, rather than the user copy/pasting the widget >> > source >> > into a cell? >> > >> > Thanks >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From tritemio at gmail.com Mon Aug 18 18:21:32 2014 From: tritemio at gmail.com (Antonino Ingargiola) Date: Mon, 18 Aug 2014 15:21:32 -0700 Subject: [IPython-dev] IPython Notebook in read-only mode In-Reply-To: <409FED76-DF37-4375-895A-A10FFE5E2C4F@gmail.com> References: <409FED76-DF37-4375-895A-A10FFE5E2C4F@gmail.com> Message-ID: Thanks to all. It turns out that, for me, the quickest solution is just make a copy of the notebooks folder and have a seconds server that launches in that dir and accepts remote connections. Even if somebody saves some modification I don't care and I can periodically overwrite the notebooks copy with my working version. Looking forward for ipython 3 with multi-user and read-only support :). Antonio On Fri, Aug 15, 2014 at 8:01 AM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > Nbviewer has an (unsupported) option at startup to serve ipynb from cwd. I > suppose that will be enough for you. > > Envoy? de mon iPhone > > > Le 14 ao?t 2014 ? 02:56, Antonino Ingargiola a > ?crit : > > > > Hi, > > > > I need to share some notebooks with some colleagues from my desktop > (windows). > > > > Would be possible to configure an ipython notebook server that accepts > connection from different machines but serves notebooks in read-only mode? > > > > I remember that at some point in the past (maybe around 0.13) I had a > configuration like this, but I don't remember if it was just some side > effect of the firewall. > > > > I could try to install nbviewer but I would also need to setup a second > server to host the ipynb files IIUC. Any guidance on how to setup nbviewer > for local files on Windows would also help :). > > > > Antonio > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hughesadam87 at gmail.com Wed Aug 20 19:32:35 2014 From: hughesadam87 at gmail.com (hugadams) Date: Wed, 20 Aug 2014 16:32:35 -0700 (PDT) Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: GREAT! Thank you On Wed, Aug 20, 2014 at 7:03 PM, Fernando Perez [via Python] < ml-node+s6n5068685h97 at n6.nabble.com> wrote: > > On Wed, Aug 20, 2014 at 3:29 PM, Adam Hughes <[hidden email] > > wrote: > >> These widgets would need to accept input data, and return output data. >> For example in the example Nicholas shared, I'd like the user to be able >> to pass in a matrix, calculate the synchronous spectra, and then use that >> for further analysis in subsequent cells in the primary notebook. Having >> that as a capability would be huge for us, because we envision a mixed >> API/GUI workflow. > > > One option here would be to provide a text field for the user to name the > variables they want returned for any given output (or you can fix those if > you want), and then have your kernel-side code store them in the user's > namespace directly. > > Here's how you can immediately write variables in ipython: > > In [7]: ip = get_ipython() > > In [8]: a = 'hello' > > In [9]: ip.user_ns['a'] > Out[9]: 'hello' > > In [10]: ip.user_ns['a'] = 100 > > In [11]: a > Out[11]: 100 > > > We can think further about whether other paradigms might be cleaner, but > I'm not sure that making variables be the return value of the widget itself > will work all that well. But in the meantime, and until we decide what the > best solution is in the long term, this will already work and it uses a > 100% stable IPython api. It may be in the end this *is* a long-term > solution we all like, but even if not, I think it's pretty serviceable. > > Cheers > > f > > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > _______________________________________________ > IPython-dev mailing list > [hidden email] > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5068685.html > To unsubscribe from Feasilibity of widgets/layout for mimicking this, click > here > > . > NAML > > -- View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5068690.html Sent from the IPython - Development mailing list archive at Nabble.com. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hughesadam87 at gmail.com Wed Aug 20 20:22:18 2014 From: hughesadam87 at gmail.com (hugadams) Date: Wed, 20 Aug 2014 17:22:18 -0700 (PDT) Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: <1408580538091-5068692.post@n6.nabble.com> Brian, Will you guys make an announcement when this new stuff is merged? If not, if you think of it, can you drop a line back to this thread so we know it's available to try out? -- View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5068692.html Sent from the IPython - Development mailing list archive at Nabble.com. From ellisonbg at gmail.com Wed Aug 20 20:36:41 2014 From: ellisonbg at gmail.com (Brian) Date: Wed, 20 Aug 2014 20:36:41 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: <1408580538091-5068692.post@n6.nabble.com> References: <1408298099961-5068197.post@n6.nabble.com> <1408580538091-5068692.post@n6.nabble.com> Message-ID: <75723EA7-161B-4B9D-A998-8901D6BE51B5@gmail.com> Ok Sent from my iPad > On Aug 20, 2014, at 8:22 PM, hugadams wrote: > > Brian, > > Will you guys make an announcement when this new stuff is merged? If not, > if you think of it, can you drop a line back to this thread so we know it's > available to try out? > > > > -- > View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5068692.html > Sent from the IPython - Development mailing list archive at Nabble.com. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From ellisonbg at gmail.com Wed Aug 20 21:12:53 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 20 Aug 2014 21:12:53 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: <75723EA7-161B-4B9D-A998-8901D6BE51B5@gmail.com> References: <1408298099961-5068197.post@n6.nabble.com> <1408580538091-5068692.post@n6.nabble.com> <75723EA7-161B-4B9D-A998-8901D6BE51B5@gmail.com> Message-ID: Oh, actually, I just remembered that we merged this PR today into master. We will probably do more work on this stuff before our release, but master should be way better than latest stable. On Wed, Aug 20, 2014 at 8:36 PM, Brian wrote: > Ok > > Sent from my iPad > >> On Aug 20, 2014, at 8:22 PM, hugadams wrote: >> >> Brian, >> >> Will you guys make an announcement when this new stuff is merged? If not, >> if you think of it, can you drop a line back to this thread so we know it's >> available to try out? >> >> >> >> -- >> View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5068692.html >> Sent from the IPython - Development mailing list archive at Nabble.com. >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From graffitici at yahoo.com Thu Aug 21 04:15:27 2014 From: graffitici at yahoo.com (graffitici at yahoo.com) Date: Thu, 21 Aug 2014 01:15:27 -0700 Subject: [IPython-dev] Using gevent for IPython CLI and notebook Message-ID: <1408608927.3778.YahooMailNeo@web164806.mail.gq1.yahoo.com> Hi, I'm developing an interactive app that uses gevent Greenlets. I want to have some tasks running in the background while users are interacting with the IPython command-line or the notebook. I realize gevent support will probably never be in the trunk, but I don't mind distributing a modified version of IPython. I've been trying to get this to work in this thread: https://github.com/ipython/ipython/issues/2785#issuecomment-52588108 adgaudio has been kind enough to help me there. But I still can't replicate what he observes. I tried it with both my system Python from homebrew, and the one in the Anaconda environment he suggested. I created the following repo: https://github.com/bbirand/gipython Can somebody quickly give it a try, and see that they get the "Expected output", as opposed to the "Observed output" that I see? I've been trying to get this for almost a month now, and I have no clue how to proceed.. Thanks! From dave.hirschfeld at gmail.com Thu Aug 21 07:21:36 2014 From: dave.hirschfeld at gmail.com (Dave Hirschfeld) Date: Thu, 21 Aug 2014 11:21:36 +0000 (UTC) Subject: [IPython-dev] ZMQ Segfault - IPython cluster Message-ID: I'm running an IPython cluster on Windows HPC x64, Python 2.7. When I start over ~80-90 engines I notice that the the controller segfaults with the only error message printed in the terminal being: Assertion failed: fds.size () <= FD_SETSIZE (bundled\zeromq\src\select.cpp:68) Looking on the net there's a fair few posts about this. My take on this is that it's a bug in libzmq - it should never just kill the host process which is exactly what the assert does (at least on windows). The below script should segfault python when N > 1023. ``` import zmq, sys N = 1024 print 'ZMQ v',zmq.zmq_version(),' N =',N ctx = zmq.Context() try: for _ in range(N): sock = ctx.socket(zmq.PAIR) sock.close() finally: del ctx ``` The problem arises with the number of connections required by the IPython cluster which it seems can easily exceed this number. I hadn't noticed this problem previously but that could well be because I wasn't using so many engines. Has anyone else come across this problem before? With 90 engines running the Task Manager reports the controller process as using 1353 handles and netstat says it has 1132 open sockets: In [44]: controller_pid = '3744' In [45]: socks = !netstat -a -o -n In [46]: len(socks) Out[46]: 2861 In [47]: len(socks.grep(controller_pid, field=-1)) Out[47]: 1132 Is this expected? If so, the default (compile time) value for FD_SETSIZE should really be increased IMHO - especially in light of the severity (segfault) of the problem. I've recompiled pyzmq/libzmq manually with FD_SETSIZE=8192 and haven't had any further problems. Regards, Dave {'commit_hash': '681fd77', 'commit_source': 'installation', 'default_encoding': 'cp1252', 'ipython_path': 'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\lib\\site- packages\\IPython', 'ipython_version': '2.1.0', 'os_name': 'nt', 'platform': 'Windows-2008ServerR2-6.1.7601-SP1', 'sys_executable': 'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\python.exe', 'sys_platform': 'win32', 'sys_version': '2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, 15:12:11) [MSC v.1500 64 bit (AMD64)]'} From Sebastien.Hinderer at inria.fr Thu Aug 21 10:55:24 2014 From: Sebastien.Hinderer at inria.fr (=?utf-8?Q?S=C3=A9bastien?= Hinderer) Date: Thu, 21 Aug 2014 16:55:24 +0200 Subject: [IPython-dev] Using IPython notebooks to teach Python Message-ID: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr> Dear all, My team currently studies the possibility to use IPython notebooks in a Python Mooc, more precisely for doing exercises. Teachers would prepare the questions in a notebook and students could answer the questions directly in the notebook itself. Actually, we would provide one copy of the original teacher's notebook for each student, so that all of them have their own notebook on which they can work independently. What I'm trying to study here is the server's security. How to prevent students from both doing malicious things on the server and accessing / modifying the notebooks of other students. Regarding the access to the server, we believe that we can secure things with AppArmor. Has such a thing aleady been done? Does anybody have experience in this area? Any help here would be greatly appreciated. What we really don't know how to do is protecting students from each other. Would it be possible to somehow run one kernel per student so that each kernel could have a different AppArmor profile associated with it? Can you think of any other approach we could use, here? Many thanks in advance for any comment, remark or suggestion. S?bastien. From benjaminrk at gmail.com Thu Aug 21 12:14:08 2014 From: benjaminrk at gmail.com (MinRK) Date: Thu, 21 Aug 2014 09:14:08 -0700 Subject: [IPython-dev] ZMQ Segfault - IPython cluster In-Reply-To: References: Message-ID: On Thu, Aug 21, 2014 at 4:21 AM, Dave Hirschfeld wrote: > I'm running an IPython cluster on Windows HPC x64, Python 2.7. > > When I start over ~80-90 engines I notice that the the controller segfaults > with the only error message printed in the terminal being: > > Assertion failed: fds.size () <= FD_SETSIZE > (bundled\zeromq\src\select.cpp:68) > > > Looking on the net there's a fair few posts about this. My take on this is > that it's a bug in libzmq - it should never just kill the host process > which > is exactly what the assert does (at least on windows). > > The below script should segfault python when N > 1023. > > ``` > import zmq, sys > N = 1024 > print 'ZMQ v',zmq.zmq_version(),' N =',N > ctx = zmq.Context() > try: > for _ in range(N): > sock = ctx.socket(zmq.PAIR) > sock.close() > finally: > del ctx > ``` > > The problem arises with the number of connections required by the IPython > cluster which it seems can easily exceed this number. > > I hadn't noticed this problem previously but that could well be because I > wasn't using so many engines. Has anyone else come across this problem > before? > > With 90 engines running the Task Manager reports the controller process as > using 1353 handles and netstat says it has 1132 open sockets: > > In [44]: controller_pid = '3744' > > In [45]: socks = !netstat -a -o -n > > In [46]: len(socks) > Out[46]: 2861 > > In [47]: len(socks.grep(controller_pid, field=-1)) > Out[47]: 1132 > > > Is this expected? If so, the default (compile time) value for FD_SETSIZE > should really be increased IMHO - especially in light of the severity > (segfault) of the problem. > > I've recompiled pyzmq/libzmq manually with FD_SETSIZE=8192 and haven't had > any further problems. > Yes, this is expected. The default for FD_SETSIZE is 1024 in libzmq (up from the default of 64 on Windows), and can be increased at compile time. As maintainer of pyzmq, I am reluctant for the defaults of pyzmq's bundled libzmq to be different from the defaults for a traditionally built libzmq, but zeromq-dev may be open to pushing up the default. -MinRK a segfault is bad memory access; assert != segfault > Regards, > Dave > > > {'commit_hash': '681fd77', > 'commit_source': 'installation', > 'default_encoding': 'cp1252', > 'ipython_path': > 'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\lib\\site- > packages\\IPython', > 'ipython_version': '2.1.0', > 'os_name': 'nt', > 'platform': 'Windows-2008ServerR2-6.1.7601-SP1', > 'sys_executable': > 'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\python.exe', > 'sys_platform': 'win32', > 'sys_version': '2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, > 15:12:11) [MSC v.1500 64 bit (AMD64)]'} > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Thu Aug 21 12:14:08 2014 From: benjaminrk at gmail.com (MinRK) Date: Thu, 21 Aug 2014 09:14:08 -0700 Subject: [IPython-dev] ZMQ Segfault - IPython cluster In-Reply-To: References: Message-ID: On Thu, Aug 21, 2014 at 4:21 AM, Dave Hirschfeld wrote: > I'm running an IPython cluster on Windows HPC x64, Python 2.7. > > When I start over ~80-90 engines I notice that the the controller segfaults > with the only error message printed in the terminal being: > > Assertion failed: fds.size () <= FD_SETSIZE > (bundled\zeromq\src\select.cpp:68) > > > Looking on the net there's a fair few posts about this. My take on this is > that it's a bug in libzmq - it should never just kill the host process > which > is exactly what the assert does (at least on windows). > > The below script should segfault python when N > 1023. > > ``` > import zmq, sys > N = 1024 > print 'ZMQ v',zmq.zmq_version(),' N =',N > ctx = zmq.Context() > try: > for _ in range(N): > sock = ctx.socket(zmq.PAIR) > sock.close() > finally: > del ctx > ``` > > The problem arises with the number of connections required by the IPython > cluster which it seems can easily exceed this number. > > I hadn't noticed this problem previously but that could well be because I > wasn't using so many engines. Has anyone else come across this problem > before? > > With 90 engines running the Task Manager reports the controller process as > using 1353 handles and netstat says it has 1132 open sockets: > > In [44]: controller_pid = '3744' > > In [45]: socks = !netstat -a -o -n > > In [46]: len(socks) > Out[46]: 2861 > > In [47]: len(socks.grep(controller_pid, field=-1)) > Out[47]: 1132 > > > Is this expected? If so, the default (compile time) value for FD_SETSIZE > should really be increased IMHO - especially in light of the severity > (segfault) of the problem. > > I've recompiled pyzmq/libzmq manually with FD_SETSIZE=8192 and haven't had > any further problems. > Yes, this is expected. The default for FD_SETSIZE is 1024 in libzmq (up from the default of 64 on Windows), and can be increased at compile time. As maintainer of pyzmq, I am reluctant for the defaults of pyzmq's bundled libzmq to be different from the defaults for a traditionally built libzmq, but zeromq-dev may be open to pushing up the default. -MinRK a segfault is bad memory access; assert != segfault > Regards, > Dave > > > {'commit_hash': '681fd77', > 'commit_source': 'installation', > 'default_encoding': 'cp1252', > 'ipython_path': > 'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\lib\\site- > packages\\IPython', > 'ipython_version': '2.1.0', > 'os_name': 'nt', > 'platform': 'Windows-2008ServerR2-6.1.7601-SP1', > 'sys_executable': > 'C:\\Python\\envs\\quantdev\\1.4.0.post673.g1d57b50\\python.exe', > 'sys_platform': 'win32', > 'sys_version': '2.7.8 |Continuum Analytics, Inc.| (default, Jul 2 2014, > 15:12:11) [MSC v.1500 64 bit (AMD64)]'} > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moritz.beber at gmail.com Thu Aug 21 12:51:28 2014 From: moritz.beber at gmail.com (Moritz Beber) Date: Thu, 21 Aug 2014 18:51:28 +0200 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr> References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr> Message-ID: Hey S?bastien, On Thu, Aug 21, 2014 at 4:55 PM, S?bastien Hinderer < Sebastien.Hinderer at inria.fr> wrote: > Dear all, > > My team currently studies the possibility to use IPython notebooks in a > Python Mooc, more precisely for doing exercises. Teachers would prepare > the questions in a notebook and students could answer the questions > directly in the notebook itself. Actually, we would provide one copy of > the original teacher's notebook for each student, so that all of them have > their own notebook on which they can work independently. > Teaching with the notebook has been discussed quite often. Search the mailing archives for more information. Of course, some of what has been said there is out-dated now. If you have the time, my recommendation would be to wait for IPython 3.0 which will come with a multi-user notebook server that should be secure to use. If you don't have the time then take a look at my repository: https://github.com/Midnighter/Notebooks-Launcher. I'm afraid it was written for Python 2 and I've not used it in a while but it lets you create user accounts for each student and then runs a separate password protected notebook server for each student in a specified directory. You can also use it to push and pull material to/from students' accounts. Naturally, you need su privileges to run it. There's also https://github.com/UnataInc/ipydra which does a similar thing but does not allow separate user accounts for each student. I talked to the author about joining efforts at some point but never got around to it. I hope these starting points help you out. Best, Moritz -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Aug 21 16:35:27 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 21 Aug 2014 13:35:27 -0700 Subject: [IPython-dev] Listing books on the home page, a policy proposal. Feedback? Message-ID: Hi folks, we just went over this on the dev meeting, but I'd like to give it a day or two here on-list for feedback in case anyone who wasn't at the meeting has further comments... For a while we've had on the homepage a link to Cyrille's IPython book, and we've been contacted by a publisher of an upcoming title asking for a new listing there. So I'd like to have uniform criteria that we use to make that decision in a fair manner with *any* publisher. Here's what we discussed during the meeting, with some additional refinements I just thought of to make things clearer. One quick note: Brian noted that the website is way too busy already, which is true. But I want to decouple this specific decision about books from an entire website redesign conversation. I think that books are one of the things that actually should belong on the main homepage, as they are a very important resource related to the project and that benefits it in multiple ways. So let's assume that any redesigned home page will have *some* space for a small image pointing to a book, and the discussion here is about what to put there and how to provide a policy publishers can all know about. Any feedback welcome. Cheers f #### Proposed language Our policy for listing books on the website is as follows. This will be uniformly applied across publishers, so the project can maintain its neutrality regarding IPython-related books (all language below would apply in the future to Jupyter-oriented books). 1. Criteria for inclusion in the homepage We will list books that are specifically and mostly about IPython itself, rather than just mentioning IPython in a chapter or two (though see Note in #2 below). For example, the currently listed book by Cyrille Rossant amply qualifies, but Wes McKinney's "Python for Data Analysis", while having an excellent chapter devoted to IPython, would not get a link on the home page. 2. Our "Books" page: We will create a main page listing in reverse chronological order the various books that are mainly devoted to IPython. For each book, we simply list its title, author and actual publisher page (or Amazon, whatever the author prefers), along with a cover image. No editorializing/reviews/etc. There may be a section at the bottom listing books that include some useful material about IPython (more than just a sentence but less than say 1/2 the book). These would be listed for reference but would have no front-page link (e.g. Wes' book). Note: the core team reserves the right to *not* include a book in this listing. If someone out there publishes a book that we deem to be absolutely terrible, we reserve the right to simply not list it. 3. What we show on the homepage We will have a single image for books, and that will be a randomly rotating screenshot of all currently available books. That image will be a link to that book's entry in the "Books" page. Since now we'll only have at most two of these (the current one and Cyrille's upcoming one), we can make the distribution give say 2/3 of views to the new book and 1/3 to the old. If eventually there's multiple books, we can figure out a *simple* distribution to highlight newer ones more often while still showing the older ones (as long as they are in print). -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From moritz.beber at gmail.com Thu Aug 21 17:11:32 2014 From: moritz.beber at gmail.com (Moritz Beber) Date: Thu, 21 Aug 2014 23:11:32 +0200 Subject: [IPython-dev] Listing books on the home page, a policy proposal. Feedback? In-Reply-To: References: Message-ID: Hey, I don't have strong opinions on this but wanted to give you my user impressions. > One quick note: Brian noted that the website is way too busy already, > which is true. But I want to decouple this specific decision about books > from an entire website redesign conversation. I think that books are one of > the things that actually should belong on the main homepage, as they are a > very important resource related to the project and that benefits it in > multiple ways. So let's assume that any redesigned home page will have > *some* space for a small image pointing to a book, and the discussion here > is about what to put there and how to provide a policy publishers can all > know about. > > In terms of website design: You probably discussed this in the meeting but I would always look for books in the documentation section. Some authors show a link right on the front page but I think that's mostly the case where the package and book author are one and the same, and it is a means of financing the project. ggplot2 did that for a while, for example. Since IPython seems nicely financed by now and the book(s) were written by third parties, I feel they don't need a prominent position on the front page. Python.org has a suggestion in the FAQ: https://docs.python.org/3/faq/general.html?highlight=book#are-there-any-books-on-python which seems less than visible to me but if I had that many books written about my project I would probably chose a similar path. So overall the docs section just seems to hit the spot for me. Best, Moritz -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Aug 21 17:48:01 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 21 Aug 2014 14:48:01 -0700 Subject: [IPython-dev] Listing books on the home page, a policy proposal. Feedback? In-Reply-To: References: Message-ID: Hi Moritz, thanks for your feedback! On Thu, Aug 21, 2014 at 2:11 PM, Moritz Beber wrote: > In terms of website design: You probably discussed this in the meeting but > I would always look for books in the documentation section. Some authors > show a link right on the front page but I think that's mostly the case > where the package and book author are one and the same, and it is a means > of financing the project. ggplot2 did that for a while, for example. Since > IPython seems nicely financed by now and the book(s) were written by third > parties, I feel they don't need a prominent position on the front page. > Python.org has a suggestion in the FAQ: > https://docs.python.org/3/faq/general.html?highlight=book#are-there-any-books-on-python > which seems less than visible to me but if I had that many books written > about my project I would probably chose a similar path. > > So overall the docs section just seems to hit the spot for me. > A couple thoughts: - good point, the "books page" could just be part of the docs page, or at the very least linked there. I'd lean towards just making it a section of the docs page for simplicity. - I think our situation regarding books is a little different from Python's: * we do get contributions from the publisher (at least so far with Packt). They have a policy of donating a part of the book's proceeds back to the project. These funds come to our Numfocus account and are unrestricted, so they are actually very useful and complementary to our more directed grants. * By now, there's probably dozens, if not over a hundred, books about Python. There's *one* IPython book, with a second in the pipeline. So I think at least at this stage, it makes sense to highlight them more prominently. If we ever find ourselves with too many, we can obviously revisit these decisions. * In our case, the books really complement our docs in a very useful way. The python docs, while not perfect, are way better than ours. Cyrille has done a great job of covering the project in a way that is very useful to newcomers, so I think it really helps our users. * Finally, I think listing official books published about the project on the site is another signal of the value of IPython to potential visitors. That shouldn't be disciounted. Based on your feedback, I'd adjust the proposal to merge 'books' into the docs page, but would leave the rest as-is, with one caveat: should we consider the policy of only listing on the home page books that contribute back to the project (like Packt's)? This would be obviously still assuming they pass our quality threshold. I can see arguments both ways, but I'd like to hear people's thoughts. Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbarratt at serialized.net Fri Aug 22 02:48:00 2014 From: jbarratt at serialized.net (Joshua Barratt) Date: Thu, 21 Aug 2014 23:48:00 -0700 Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython Notebook on your own machine In-Reply-To: References: Message-ID: It *almost* works on IPython 2.x, except the nbserver-* JSON blobs only have the PID's in the name, not the actual file. I took a pass at fixing this up for IPython 2.x, by borrowing some of the code from the master branch, and it seems to be working. https://gist.github.com/jbarratt/ae8026493fedc79f122b On Sun, Aug 3, 2014 at 10:49 PM, Thomas Kluyver wrote: > Neat, thanks for those. For nblist, instead of scanning processes and > ports, it may be easier to use our API function list_running_servers(), > especially for cross platform support: > > > http://ipython.org/ipython-doc/dev/api/generated/IPython.html.notebookapp.html#IPython.html.notebookapp.list_running_servers > > nbmanager also includes a slightly higher level API around that, which is > completely independent of all the GUI code (though this requires IPython 3): > > https://github.com/takluyver/nbmanager/blob/master/nbmanager/api.py > > Best wishes, > Thomas > > > On 3 August 2014 22:34, Joshua Barratt wrote: > >> Thanks for sharing those, very cool. I had been thinking about doing >> something like nbopen, I'm happy to see that's something I can delete off >> of the infinite to-do list. >> >> I created something similar to nbmanager, but much more stripped down and >> only allows viewing. >> >> I called mine nblist >> . >> >> $ nblist >> http://127.0.0.1:8088 | /Users/me/work/notebooks >> >> http://127.0.0.1:8089 | /Users/me/work/otherproject >> >> >> I also recently wrote something called nbgrep >> (blog post here >> ) >> which allows you to search the input cells of all your notebooks, >> regardless of directory. >> >> Unfortunately both of mine only work on OSX, though they could be >> modified to be cross-platform. >> >> Josh >> >> On Fri, Aug 1, 2014 at 2:55 PM, Thomas Kluyver wrote: >> >>> I've been working on a couple of tools to improve the experience of >>> working with IPython notebooks on your own computer. >>> >>> First, nbopen, a tool to open a notebook in the nearest available >>> server, or start a new server if there isn't one available. It now >>> integrates with your file manager so you can double click on a notebook >>> file to open it (open source desktops only for now, PRs for other systems >>> welcome): >>> >>> https://github.com/takluyver/nbopen >>> >>> However, there's no easy way to shut down notebook servers launched by >>> double clicking on notebooks, because there isn't a terminal where you can >>> press ctrl-c. So I also made nbmanager, a GUI app which discovers running >>> notebook servers and open notebooks, and lets you easily shut them down: >>> >>> https://github.com/takluyver/nbmanager >>> >>> nbopen will work on IPython 2, while nbmanager, which is more >>> experimental, requires you to be running IPython from git at the moment. >>> >>> Thomas >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Sebastien.Hinderer at inria.fr Fri Aug 22 02:48:02 2014 From: Sebastien.Hinderer at inria.fr (=?utf-8?Q?S=C3=A9bastien?= Hinderer) Date: Fri, 22 Aug 2014 08:48:02 +0200 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr> Message-ID: <20140822064802.GA17770@pl-59055.rocqadm.inria.fr> Dear Moritz, Manyh thanks for your prompt and helpfll response! > Teaching with the notebook has been discussed quite often. Search the > mailing archives for more information. Of course, some of what has been > said there is out-dated now. If you have the time, my recommendation would > be to wait for IPython 3.0 which will come with a multi-user notebook > server that should be secure to use. If you don't have the time then take a > look at my repository: > https://github.com/Midnighter/Notebooks-Launcher. Well unless iPython 3.0 is released very, very soon, I think we don't have the time to wait for it so I'll definitely have a look to your repository, thanks! Is there any planned date for the release of iPython 3.0? > I'm afraid it was written for Python 2 and I've not used it in a while but > it lets you create user accounts for each student and then runs a separate > password protected notebook server for each student in a specified > directory. This seems a very interesting starting point for expeimenting! > You can also use it to push and pull material to/from students' > accounts. Naturally, you need su privileges to run it. Sure. I presume it will be the sme with iPython 3.0? Will the server have to be run as root and then, once a user logs in, the serverwill continue running as that user? So does it mean that users will need to have a full Unix account on the server? > There's also https://github.com/UnataInc/ipydra which does a similar thing > but does not allow separate user accounts for each student. I talked to the > author about joining efforts at some point but never got around to it. Will have a look to that, too! Thanks! > I hope these starting points help you out. They definitely do, thanks. Any comment on the AppArmor aspect would also be helpful. Best wishes and thanks again, S?bastien. From moritz.beber at gmail.com Fri Aug 22 04:41:31 2014 From: moritz.beber at gmail.com (Moritz Beber) Date: Fri, 22 Aug 2014 10:41:31 +0200 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: <20140822064802.GA17770@pl-59055.rocqadm.inria.fr> References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr> <20140822064802.GA17770@pl-59055.rocqadm.inria.fr> Message-ID: Hi, > Well unless iPython 3.0 is released very, very soon, I think we don't > have the time to wait for it so I'll definitely have a look to your > repository, thanks! Is there any planned date for the release of iPython > 3.0? > According to this: https://github.com/ipython/ipython/wiki/Roadmap:-IPython fairly soon but any dev will be able to provide a better answer. > > > I'm afraid it was written for Python 2 and I've not used it in a while > but > > it lets you create user accounts for each student and then runs a > separate > > password protected notebook server for each student in a specified > > directory. > > This seems a very interesting starting point for expeimenting! > > > You can also use it to push and pull material to/from students' > > accounts. Naturally, you need su privileges to run it. > > Sure. I presume it will be the sme with iPython 3.0? Will the server > have to be run as root and then, once a user logs in, the serverwill > continue running as that user? So does it mean that users will need to > have a full Unix account on the server? > I'm not familiar with the plans for the multi-user server in 3.0. It's possible that the users will simply be distinguished by login and thus restricted in their rights but that's just guessing on my part now. What my script does is create a full Unix account for each student and then start a separate password protected notebook server for each student. It then spawns a simple website with links for each student to the appropriate address/port of their notebook server. > Any comment on the AppArmor aspect would also be helpful. > > I have zero experience with AppArmor. All I can say is that the users are running their notebook under their unix user account. So as long as the unix system is set up properly they should not be able to do any harm. Furthermore the notebooks are password protected so unless a student chooses to grant the necessary rights or hands out his password, others cannot affect it. I have also always clearly announced at the beginning of a course that a student who tries to mess with the unix system or other people's stuff will (a) fail the course and (b) be brought before a committee for academic conduct. This was obviously at a university so I don't know if this is relevant to you. Good luck and feel free to mail me off list or open issues on github if you have problems with my script. Moritz -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Fri Aug 22 05:57:13 2014 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Fri, 22 Aug 2014 11:57:13 +0200 Subject: [IPython-dev] Listing books on the home page, a policy proposal. Feedback? In-Reply-To: References: Message-ID: > * By now, there's probably dozens, if not over a hundred, books about Python. There's *one* IPython book, with a second in the pipeline. So I think at least at this stage, it makes sense to highlight them more prominently. If we ever find ourselves with too many, we can obviously revisit these decisions. Just to be clear and have it written somewhere : What about book written as notebook that do not speak about IPython in particular ? (assuming the notebook are available to be re-executed of course) -- M From Sebastien.Hinderer at inria.fr Fri Aug 22 05:55:58 2014 From: Sebastien.Hinderer at inria.fr (=?utf-8?Q?S=C3=A9bastien?= Hinderer) Date: Fri, 22 Aug 2014 11:55:58 +0200 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr> <20140822064802.GA17770@pl-59055.rocqadm.inria.fr> Message-ID: <20140822095558.GA5722@pl-59055.rocqadm.inria.fr> Dear Moritz, I have just discovered this project: https://github.com/cni/ipython-hydra Which seems even closer to yours than ipydra. Are you able to say what differs between your project and this one? Is it just a matter of one having a web interface and the other not, or can you see other differences? Many thanks! S?bastien. From Sebastien.Hinderer at inria.fr Fri Aug 22 06:06:21 2014 From: Sebastien.Hinderer at inria.fr (=?utf-8?Q?S=C3=A9bastien?= Hinderer) Date: Fri, 22 Aug 2014 12:06:21 +0200 Subject: [IPython-dev] Adding support for more cell types? Message-ID: <20140822100621.GA32228@pl-59055.rocqadm.inria.fr> Dear all, Another question we have, at a longer term, though, would be to figure out whether iPython could be used to deal with notebooks containing code blocks for other programming languages. How easy would it be to add, say, suport for Haskell, OCaml or Lisp cells? Many thanks in advance for any pointer! Best wishes, S?bastien. From bussonniermatthias at gmail.com Fri Aug 22 06:16:52 2014 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Fri, 22 Aug 2014 12:16:52 +0200 Subject: [IPython-dev] Adding support for more cell types? In-Reply-To: <20140822100621.GA32228@pl-59055.rocqadm.inria.fr> References: <20140822100621.GA32228@pl-59055.rocqadm.inria.fr> Message-ID: <2554C7B4-A304-4B22-A69D-714BD7718F66@gmail.com> Le 22 ao?t 2014 ? 12:06, S?bastien Hinderer a ?crit : > Dear all, > > Another question we have, at a longer term, though, would be to figure > out whether iPython could be used to deal with notebooks containing code > blocks for other programming languages. How easy would it be to add, > say, suport for Haskell, OCaml or Lisp cells? Haskell and OCamel already ave full kernel implementation : https://github.com/ipython/ipython/wiki/Projects-using-IPython You are welcome to create a Lisp on. If you want to mix/match language you could create a custom magic: (see cython one for ex [1] ) Or look at calico (above link) -- M [1] : http://nbviewer.ipython.org/github/ipython/ipython/blob/3607712653c66d63e0d7f13f073bde8c0f209ba8/docs/examples/notebooks/cython_extension.ipynb -------------- next part -------------- An HTML attachment was scrubbed... URL: From gvwilson at third-bit.com Fri Aug 22 06:51:44 2014 From: gvwilson at third-bit.com (Greg Wilson) Date: Fri, 22 Aug 2014 06:51:44 -0400 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: <20140822095558.GA5722@pl-59055.rocqadm.inria.fr> References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr> <20140822064802.GA17770@pl-59055.rocqadm.inria.fr> <20140822095558.GA5722@pl-59055.rocqadm.inria.fr> Message-ID: <53F720C0.5010903@third-bit.com> Hi all, We have been using the IPython Notebook to teach Software Carpentry classes for a year now, and while we like it a lot, we've been bumping into two issues: 1. For most students, the notebook is an entirely new kind of interface unlike any they've seen before, which means they're trying to learn two things at once: Python, and the notebook itself. I've had good success introducing Python first via the standard command-line interpreter, letting them get comfortable with a few basic ideas (say, from "assign a value to a variable" to "the code block belonging to a loop must be indented"), and *then* introducing the notebook. This fall, I'm going to experiment with showing them Python via Wing101 (a simple IDE that's widely used in undergrad computer science teaching), then switching to the notebook. 2. Notebooks and version control don't play nicely together. This may not be an issue for you if you're not trying to teach students to use version control as part of the class, but diffing and merging the source of notebooks is out of reach for novice programmers. Cheers, Greg On 2014-08-22 5:55 AM, S?bastien Hinderer wrote: > Dear Moritz, > > I have just discovered this project: > > https://github.com/cni/ipython-hydra > > Which seems even closer to yours than ipydra. > > Are you able to say what differs between your project and this one? > Is it just a matter of one having a web interface and the other not, or > can you see other differences? > > Many thanks! > S?bastien. > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From moritz.beber at gmail.com Fri Aug 22 08:12:54 2014 From: moritz.beber at gmail.com (Moritz Beber) Date: Fri, 22 Aug 2014 14:12:54 +0200 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: <20140822095558.GA5722@pl-59055.rocqadm.inria.fr> References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr> <20140822064802.GA17770@pl-59055.rocqadm.inria.fr> <20140822095558.GA5722@pl-59055.rocqadm.inria.fr> Message-ID: > > I have just discovered this project: > > https://github.com/cni/ipython-hydra > > Which seems even closer to yours than ipydra. > > Are you able to say what differs between your project and this one? > Is it just a matter of one having a web interface and the other not, or > can you see other differences? > > The main difference is explained in one of their paragraphs: "addldapuser will add a new user to the system, if they don't already exist. It must be run as root. The password is disabled because we use Stanford's kerberos authentication, so if users log in via ssh they can authenticate with their SUNet ID and password. The new user is also added to the ipython group. If the user exists in Stanford LDAP, then their UID will be assigned by the LDAP result. Otherwise, a local UID is used." They integrated it with an LDAP server and that is something I shied away from. The integration with that LDAP server is hard coded into their bash scripts so it would have taken me some time to decouple it. On top of that I needed to run the server both on Mac and Linux, so it was easier for me to use Python for that cross-platform compatibility. Another difference is that they spawn notebooks and then check the port assigned whereas my script assigns the port deterministically (but does not actually check whether that port was already in use before - so that's desperately needed). So assigning ports beforehand (with the proper checks) might be nicer when you need to open firewall ports only in a certain range. I wouldn't mind improving the whole script to update it to the new IPython 2.0 and make it Python 3.0 compatible, and use a simple sqlite database. I have time for that after September 8. If that's not too late for you, we could then also talk about specific problems and feature requests. Cheers -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug.blank at gmail.com Fri Aug 22 08:23:53 2014 From: doug.blank at gmail.com (Doug Blank) Date: Fri, 22 Aug 2014 08:23:53 -0400 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr> <20140822064802.GA17770@pl-59055.rocqadm.inria.fr> <20140822095558.GA5722@pl-59055.rocqadm.inria.fr> Message-ID: You may also want to follow the development of the Jupyter Hub for IPython3: https://github.com/jupyter/jupyterhub Although I suspect that IPython3/Jupyter won't be released for months, I was able to login (via my Linux id and password) with the code as of yesterday. In addition, I was able to use our new IPython3 kernels without any obvious issues. It looks like it will be very easy to substitute the os authentication for another. I'm making plans to use this multi-user hub in the classroom as soon as possible this semester. (I'm teaching Programming Languages, and so easily switching between languages should make an interesting basis for the course.) -Doug On Fri, Aug 22, 2014 at 8:12 AM, Moritz Beber wrote: > > >> I have just discovered this project: >> >> https://github.com/cni/ipython-hydra >> >> Which seems even closer to yours than ipydra. >> >> Are you able to say what differs between your project and this one? >> Is it just a matter of one having a web interface and the other not, or >> can you see other differences? >> >> > The main difference is explained in one of their paragraphs: > > "addldapuser will add a new user to the system, if they don't already > exist. It must be run as root. The password is disabled because we use > Stanford's kerberos authentication, so if users log in via ssh they can > authenticate with their SUNet ID and password. The new user is also added > to the ipython group. If the user exists in Stanford LDAP, then their UID > will be assigned by the LDAP result. Otherwise, a local UID is used." > > They integrated it with an LDAP server and that is something I shied away > from. The integration with that LDAP server is hard coded into their bash > scripts so it would have taken me some time to decouple it. On top of that > I needed to run the server both on Mac and Linux, so it was easier for me > to use Python for that cross-platform compatibility. > > Another difference is that they spawn notebooks and then check the port > assigned whereas my script assigns the port deterministically (but does not > actually check whether that port was already in use before - so that's > desperately needed). So assigning ports beforehand (with the proper checks) > might be nicer when you need to open firewall ports only in a certain range. > > I wouldn't mind improving the whole script to update it to the new IPython > 2.0 and make it Python 3.0 compatible, and use a simple sqlite database. I > have time for that after September 8. If that's not too late for you, we > could then also talk about specific problems and feature requests. > > Cheers > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.hirschfeld at gmail.com Fri Aug 22 10:26:11 2014 From: dave.hirschfeld at gmail.com (Dave Hirschfeld) Date: Fri, 22 Aug 2014 14:26:11 +0000 (UTC) Subject: [IPython-dev] ZMQ Segfault - IPython cluster References: Message-ID: MinRK gmail.com> writes: > > > Yes, this is expected. The default for FD_SETSIZE is 1024 in libzmq (up from the default of 64 on Windows), and can be increased at compile time. As maintainer of pyzmq, I am reluctant for the defaults of pyzmq's bundled libzmq to be different from the defaults for a traditionally built libzmq, but zeromq-dev may be open to pushing up the default. > > > -MinRK > ? > > a segfault is bad memory access; assert != segfault > It was certainly unexpected for me! :P There's also no mention of FD_SETSIZE on either the IPython or PyZMQ websites AFAICS. It was a fairly nasty bug to track down because the remote process failed with no indication of why it did so. This is because Windows HPC 2008 only caches the first 4000 characters of stdout/stderr so the error message never got displayed (this has changed to a much more useful last 4k chars in 2012). To actually see the message I had to remote on the node and start the ipcontroller process manually in a terminal. After actually managing to get an error message I then had to search the internet for what it meant and then recompile libzmq itself with VS2008 which isn't exactly a cakewalk for the uninitiated. I agree it should be fixed in the upstream library so I've opened an issue there - we can see what they say there... https://github.com/zeromq/libzmq/issues/1165 Thanks, Dave From doug.blank at gmail.com Fri Aug 22 11:46:47 2014 From: doug.blank at gmail.com (Doug Blank) Date: Fri, 22 Aug 2014 11:46:47 -0400 Subject: [IPython-dev] Just for fun: chatbot in Jupyter/IPython3 Message-ID: Just testing out the new ipython-master and jupyterhub repositories. If you are trying those, you can easily add a chatbot kernel: $ pip install chatbot_kernel Now you can use a console with chatbot_kernel, or select "Chatbot" from the notebook. Perhaps this type of kernel could actually be made into a help-bot, or tutor. Or just to have some fun :) $ ipython console --kernel chatbot_kernel (loads after a few seconds; loading databases...) (hit enter to display messages; need to flush) In [1]: Hello Hi there! In [2]: Who are you? Nameless In [3]: What time is it? The time here is Fri Aug 22 11:35:00 2014 In [4]: Do you like movies? Yes, I love film, especially science-fiction and comedy. In [5]: What is your favorite movie? My favorite movie is . Have you seen ? In [6]: You have some bugs What makes you think I have bugs? In [7]: Do you have a mother? No I only have a father. In [8]: What is your father's name? My is . In [9]: I have a fear of spiders Interesting. I have a lot of answers to silly questions. In [10]: What do you want to do? I would like to study psychology and maybe go into law or politics. In [11]: I like computer science What do you think of the way the press covers your field? In [12]: It is mostly hype Really. mostly hype is? In [13]: Have you heard of Jupyter? No, tell me about him, her or it. In [14]: It is a system for computing mostly hype is a system for computing. I suppose that makes sense. In [15]: I see you take my words and re use them Yes I take his words and re use them. In [16]: Ok, goodbye "Yes I take his words and re use them."? Ok, goodbye?. Sayonara. -Doug PS - source is at: https://bitbucket.org/ipre/calico/src/master/src/chatbot_kernel/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From damianavila at gmail.com Fri Aug 22 11:49:03 2014 From: damianavila at gmail.com (=?UTF-8?Q?Dami=C3=A1n_Avila?=) Date: Fri, 22 Aug 2014 12:49:03 -0300 Subject: [IPython-dev] Just for fun: chatbot in Jupyter/IPython3 In-Reply-To: References: Message-ID: Ja ja... nice!!! 2014-08-22 12:46 GMT-03:00 Doug Blank : > Just testing out the new ipython-master and jupyterhub repositories. If > you are trying those, you can easily add a chatbot kernel: > > $ pip install chatbot_kernel > > Now you can use a console with chatbot_kernel, or select "Chatbot" from > the notebook. Perhaps this type of kernel could actually be made into a > help-bot, or tutor. Or just to have some fun :) > > $ ipython console --kernel chatbot_kernel > (loads after a few seconds; loading databases...) > (hit enter to display messages; need to flush) > > In [1]: Hello > Hi there! > > In [2]: Who are you? > Nameless > > In [3]: What time is it? > The time here is Fri Aug 22 11:35:00 2014 > > In [4]: Do you like movies? > Yes, I love film, especially science-fiction and comedy. > > In [5]: What is your favorite movie? > My favorite movie is . Have you seen ? > > In [6]: You have some bugs > What makes you think I have bugs? > > In [7]: Do you have a mother? > No I only have a father. > > In [8]: What is your father's name? > My is . > > In [9]: I have a fear of spiders > Interesting. I have a lot of answers to silly questions. > > In [10]: What do you want to do? > I would like to study psychology and maybe go into law or politics. > > In [11]: I like computer science > What do you think of the way the press covers your field? > > In [12]: It is mostly hype > Really. mostly hype is? > > In [13]: Have you heard of Jupyter? > No, tell me about him, her or it. > > In [14]: It is a system for computing > mostly hype is a system for computing. I suppose that makes sense. > > In [15]: I see you take my words and re use them > Yes I take his words and re use them. > > In [16]: Ok, goodbye > "Yes I take his words and re use them."? Ok, goodbye?. Sayonara. > > -Doug > > PS - source is at: > https://bitbucket.org/ipre/calico/src/master/src/chatbot_kernel/ > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- *Dami?n* -------------- next part -------------- An HTML attachment was scrubbed... URL: From ocefpaf at gmail.com Fri Aug 22 11:55:48 2014 From: ocefpaf at gmail.com (Filipe Pires Alvarenga Fernandes) Date: Fri, 22 Aug 2014 12:55:48 -0300 Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter Message-ID: Have anyone tried to use venvs as kernels? Making the installed vens show up in the drop-down menu in the new Jupyter Notebook? I was toying with the idea but got nowhere near something useful. I think it would be a nice way to switch between virtual environments. -Filipe -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Aug 22 13:34:04 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 22 Aug 2014 10:34:04 -0700 Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter In-Reply-To: References: Message-ID: We looked into it and as far as we can tell, all the pieces we're providing for the kernel spec should make it possible to have your kernel entry point do the switching. But we will NOT bake into the kernel spec direct knowledge of venv, conda envs, or any other 'enviornment manager'. We want to make it *possible* do declare kernels that are connected to environments, but as of yet, there are just too many ways of 'managing environments' across languages and toolchains for us to enshrine any particular approach into the spec itself. As I say above, we looked into it, but we didn't actually implement anything yet. So we'd encourage you to try things out, and report both with what works (I'm sure others are interested) and what doesn't (so we can help out or fix things if needed) Cheers f On Fri, Aug 22, 2014 at 8:55 AM, Filipe Pires Alvarenga Fernandes < ocefpaf at gmail.com> wrote: > Have anyone tried to use venvs as kernels? Making the installed vens show > up in the drop-down menu in the new Jupyter Notebook? > > I was toying with the idea but got nowhere near something useful. > > I think it would be a nice way to switch between virtual environments. > > -Filipe > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Aug 22 13:36:01 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 22 Aug 2014 10:36:01 -0700 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: References: <20140821145524.GA15744@pl-59055.rocqadm.inria.fr> <20140822064802.GA17770@pl-59055.rocqadm.inria.fr> <20140822095558.GA5722@pl-59055.rocqadm.inria.fr> Message-ID: On Fri, Aug 22, 2014 at 5:23 AM, Doug Blank wrote: > You may also want to follow the development of the Jupyter Hub for > IPython3: > > https://github.com/jupyter/jupyterhub > BTW, that's what we mean when we say "the multiuser IPython server". Since that machinery is fully language agnostic, we're putting it as part of Jupyter itself from the get-go. I know Doug knows this, I just want to make sure it's clear for everyone else... > Although I suspect that IPython3/Jupyter won't be released for months, I > was able to login (via my Linux id and password) with the code as of > yesterday. In addition, I was able to use our new IPython3 kernels without > any obvious issues. It looks like it will be very easy to substitute the os > authentication for another. > We're hoping to have a release by October/November. I actually think that's pretty realistic. It may not have too many bells and whistles, but given that Min's current prototype already does work, we should be OK pushing out a basic MVP in that timeframe. Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Aug 22 13:36:49 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 22 Aug 2014 10:36:49 -0700 Subject: [IPython-dev] Just for fun: chatbot in Jupyter/IPython3 In-Reply-To: References: Message-ID: On Fri, Aug 22, 2014 at 8:46 AM, Doug Blank wrote: > Just testing out the new ipython-master and jupyterhub repositories. If > you are trying those, you can easily add a chatbot kernel: > > $ pip install chatbot_kernel > This is just awesome! Thanks :) f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Fri Aug 22 14:35:42 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 22 Aug 2014 14:35:42 -0400 Subject: [IPython-dev] Just for fun: chatbot in Jupyter/IPython3 In-Reply-To: References: Message-ID: <8489708C-68D9-4C9E-AB75-060C7CBD0CC8@gmail.com> :-) Sent from my iPhone > On Aug 22, 2014, at 11:46 AM, Doug Blank wrote: > > Just testing out the new ipython-master and jupyterhub repositories. If you are trying those, you can easily add a chatbot kernel: > > $ pip install chatbot_kernel > > Now you can use a console with chatbot_kernel, or select "Chatbot" from the notebook. Perhaps this type of kernel could actually be made into a help-bot, or tutor. Or just to have some fun :) > > $ ipython console --kernel chatbot_kernel > (loads after a few seconds; loading databases...) > (hit enter to display messages; need to flush) > > In [1]: Hello > Hi there! > > In [2]: Who are you? > Nameless > > In [3]: What time is it? > The time here is Fri Aug 22 11:35:00 2014 > > In [4]: Do you like movies? > Yes, I love film, especially science-fiction and comedy. > > In [5]: What is your favorite movie? > My favorite movie is . Have you seen ? > > In [6]: You have some bugs > What makes you think I have bugs? > > In [7]: Do you have a mother? > No I only have a father. > > In [8]: What is your father's name? > My is . > > In [9]: I have a fear of spiders > Interesting. I have a lot of answers to silly questions. > > In [10]: What do you want to do? > I would like to study psychology and maybe go into law or politics. > > In [11]: I like computer science > What do you think of the way the press covers your field? > > In [12]: It is mostly hype > Really. mostly hype is? > > In [13]: Have you heard of Jupyter? > No, tell me about him, her or it. > > In [14]: It is a system for computing > mostly hype is a system for computing. I suppose that makes sense. > > In [15]: I see you take my words and re use them > Yes I take his words and re use them. > > In [16]: Ok, goodbye > "Yes I take his words and re use them."? Ok, goodbye?. Sayonara. > > -Doug > > PS - source is at: > https://bitbucket.org/ipre/calico/src/master/src/chatbot_kernel/ > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From ocefpaf at gmail.com Fri Aug 22 14:40:39 2014 From: ocefpaf at gmail.com (Filipe Pires Alvarenga Fernandes) Date: Fri, 22 Aug 2014 15:40:39 -0300 Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter In-Reply-To: References: Message-ID: On Fri, Aug 22, 2014 at 2:34 PM, Fernando Perez wrote: > We looked into it and as far as we can tell, all the pieces we're > providing for the kernel spec should make it possible to have your kernel > entry point do the switching. > OK, I will keep trying to have something more robust. Right now I am doing a very hackish workaround. > But we will NOT bake into the kernel spec direct knowledge of venv, conda > envs, or any other 'enviornment manager'. We want to make it *possible* do > declare kernels that are connected to environments, but as of yet, there > are just too many ways of 'managing environments' across languages and > toolchains for us to enshrine any particular approach into the spec itself. > I understand (and I agree) with that. > As I say above, we looked into it, but we didn't actually implement > anything yet. So we'd encourage you to try things out, and report both with > what works (I'm sure others are interested) and what doesn't (so we can > help out or fix things if needed) > As soon as I have something better I will post it here. > Cheers > > f > Thanks, -Filipe > > > On Fri, Aug 22, 2014 at 8:55 AM, Filipe Pires Alvarenga Fernandes < > ocefpaf at gmail.com> wrote: > >> Have anyone tried to use venvs as kernels? Making the installed vens >> show up in the drop-down menu in the new Jupyter Notebook? >> >> I was toying with the idea but got nowhere near something useful. >> >> I think it would be a nice way to switch between virtual environments. >> >> -Filipe >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rsarava at gmail.com Fri Aug 22 20:24:33 2014 From: rsarava at gmail.com (Ramalingam Saravanan) Date: Fri, 22 Aug 2014 19:24:33 -0500 Subject: [IPython-dev] Using IPython notebooks to teach Python Message-ID: Hi S?bastien, The multi-user IPython server would be the best option for your needs, but here are some alternatives: https://github.com/ptone/jiffylab https://cloud.sagemath.com https://github.com/mitotic/graphterm (my own project; somewhat experimental) Here's a paper from SciPy14 proceedings that discusses these alternatives: https://dl.dropboxusercontent.com/u/72208800/docs/Saravanan-SciPy14.pdf Best, Saravanan On Thu, Aug 21, 2014 at 4:55 PM, S?bastien Hinderer < Sebastien.Hinderer at inria.fr> wrote: > Dear all, > > My team currently studies the possibility to use IPython notebooks in a > Python Mooc, more precisely for doing exercises. Teachers would prepare > the questions in a notebook and students could answer the questions > directly in the notebook itself. Actually, we would provide one copy of > the original teacher's notebook for each student, so that all of them have > their own notebook on which they can work independently. > > What I'm trying to study here is the server's security. How to prevent > students from both doing malicious things on the server and accessing / > modifying the notebooks of other students. -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sat Aug 23 01:12:59 2014 From: benjaminrk at gmail.com (MinRK) Date: Fri, 22 Aug 2014 22:12:59 -0700 Subject: [IPython-dev] IPython3/Jupyter Magics for Python-based kernels In-Reply-To: References: Message-ID: I think if you want to add something like magics for Python-wrapped kernels based on your magickernel.py code, that would be interesting. Standardizing magics across kernel languages is something that is attractive on its face, but we deemed outside the scope of the message / kernel specification, because it is probably not feasible to do in a language agnostic way (For example, IPython magics can contain Python string-formatting code with Python expressions that are evaluated in the interactive namespace). Clarification of discussions like this are part of the reasoning behind the Jupyter/IPython split: - Jupyter is language-agnostic computing tools, such as the message protocol, notebook document format, and notebook UI. - IPython is a better interactive Python environment, with some extensions to the Python language for interactive use. - IPython provides a Jupyter Kernel. - Magics are 100% an IPython feature, and not in any way related to Jupyter. - Other kernels are welcome to implement extensions to their language (including Python!) as they see fit, as similar or different to IPython?s magics system as makes sense. Implementing magics for basic Python-wrapped kernels (i.e. not R, Julia, Ruby, Haskell, etc.) would be simple, and potentially useful, depending on how many of these kernels are developed. I do not expect most first-class kernels to go this route, in which case they would not share any magic code. Further, I don?t think the magic code developed for this use case would even be shared by IPython. But it would still be an interesting exercise, and might encourage other kernels to take the multi-interpreter approach if it is successful. Sounds like a fun experiment! -MinRK ? On Wed, Aug 20, 2014 at 4:22 AM, Doug Blank wrote: > IPython developers, > > One of the nice things that the IPython team did for the next version was > to create a base class for creating new kernels for code written in Python, > called wrapper kernels [1]. > > Unfortunately, such kernels don't have ! shell commands, nor magics. This > is understandable, as those are tightly integrated with IPython's Python. > However, one could get some of the basic functionality of these features on > top of the wrapper kernel, in a generic way. For example, you could run > shell commands, or have some magics that operate independent of the kernel. > > We have created a prototype of magics for wrapper kernels [2]. The code is > in magickernel.py, and the magics are in the folder magics. The current > prototype works like this: > > ``` > In [1]: %magics > Line magics: > %cd PATH - change current directory of session > %connect_info - show connection information > %download URL [FILENAME] - download file from URL > %html CODE - display code as HTML > %install_magic URL - download and install magic from URL > %javascript CODE - send code as JavaScript > %latex TEXT - display text as LaTex > %magics - show installed magics > %reload_magics - reload the magics from the installed files > %shell COMMAND - run the line as a shell command > %time COMMAND - show time to run line > > Cell magics: > %%file FILENAME - write contents of cell to file > %%html - display contents of cell as HTML > %%javascript - send contents of cell as JavaScript > %%latex - display contents of cell as LaTeX > %%shell - run the contents of the cell as shell commands > %%time - show time to run cell > > Shell shortcut: > ! COMMAND ... - execute command in shell > > Any cell magic can be made persistent for rest of session by using %%% > prefix. > ``` > > The idea is to be able to easily add some standard-like magics to > 3rd-party kernels by using MagicKernel. See the sample wrapperkernel > EchoKernel with magics [3]. Also, the idea here is not to replicate too > much what comes with IPython's Python, but to provide a small, simple core > set of magics that can't be easily done in the kernel language. You can > install MagicKernel from source, or using: > > $ pip install calico --upgrade > > This is a prototype (with some bugs), and we very much hope for others to > provide feedback or code, and more work needs to be done. What magics would > you need in your 3rd-party kernel, that can be accomplished this way? Will > this design be sufficient? > > We hope that this is useful! > > -Doug > > [1] - http://ipython.org/ipython-doc/dev/development/wrapperkernels.html > [2] - https://bitbucket.org/ipre/calico/src/master/src/calico/ > [3] - > https://bitbucket.org/ipre/calico/src/master/src/echo_kernel/echo_kernel.py > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From claresloggett at gmail.com Sat Aug 23 07:30:55 2014 From: claresloggett at gmail.com (Clare Sloggett) Date: Sat, 23 Aug 2014 21:30:55 +1000 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: References: Message-ID: Wakari (wakari.io) is another free service with fairly quick sign-up. I've used Wakari and SageMathCloud as backup options for when a student had an issue with the uni infrastructure. My understanding is that these services manage security is to provide each user with their own virtualised environment so they can't really do any harm, and that this can be something lightweight, like docker. If you're going to use a service outside your institution you might want to watch out for websocket proxying issues.. I've run into these problems in some places! On 23 August 2014 10:24, Ramalingam Saravanan wrote: > Hi S?bastien, > > The multi-user IPython server would be the best option for your needs, but > here are some alternatives: > > https://github.com/ptone/jiffylab > > https://cloud.sagemath.com > > https://github.com/mitotic/graphterm (my own project; somewhat > experimental) > > Here's a paper from SciPy14 proceedings that discusses these alternatives: > > > https://dl.dropboxusercontent.com/u/72208800/docs/Saravanan-SciPy14.pdf > > Best, > Saravanan > > > On Thu, Aug 21, 2014 at 4:55 PM, S?bastien Hinderer < > Sebastien.Hinderer at inria.fr> wrote: > > > Dear all, > > > > My team currently studies the possibility to use IPython notebooks in a > > Python Mooc, more precisely for doing exercises. Teachers would prepare > > the questions in a notebook and students could answer the questions > > directly in the notebook itself. Actually, we would provide one copy of > > the original teacher's notebook for each student, so that all of them > have > > their own notebook on which they can work independently. > > > > What I'm trying to study here is the server's security. How to prevent > > students from both doing malicious things on the server and accessing / > > modifying the notebooks of other students. > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cyrille.rossant at gmail.com Sat Aug 23 12:50:20 2014 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Sat, 23 Aug 2014 18:50:20 +0200 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: Adam, FYI, you'll find my EuroSciPy tutorial on the widgets here Best, Cyrille 2014-08-20 10:15 GMT+02:00 Cyrille Rossant : > Hi Adam, > > I'll do a tutorial about IPython widgets at EuroSciPy in a week [1]. > I'm notably planning to demonstrate how to build, in the notebook, a > GUI like the one you showed. I'll put here the link to the GitHub repo > when everything's ready. > > Best, > Cyrille > > [1] https://www.euroscipy.org/2014/schedule/presentation/60/ > > 2014-08-17 19:54 GMT+02:00 hugadams : >> (Please disregard if doublepost; having some subscription issues) >> >> Hello, >> >> We are developing a spectroscopy library and want to put in some interactive >> plots/utilities to make the software a bit more approachable to novices. >> I'd really like to build the entire interactive plotting interface through >> the notebook's new widget system. My reasoning is as follows: >> >> - All static plots are already matplotlib-based, so we wouldn't have to >> write a seperate plotting api >> - Notebooks are inherently browser-ready, so sharing plots would be great >> - Is it likely that static nbviewer will eventually support >> widgets? Since the data is required, I figured not... >> - Our tutorials are already in notebooks, so we'd like to be consistent. >> >> I've watched all the videos I can find from the widgets, and think I'm fine >> with the Python end; however, am finding it hard to come by lots of >> information on controlling the layout of the widgets. I know it's mostly >> done in javascript, and I think the most complex view we will have will look >> something like this: >> >> https://sites.google.com/site/shigemorita/home/2dshige >> >> How much time would you estimate a javascript novice ought to spend on >> getting something like that in the notebook? Can anyone share tutorials for >> the layout portion of the widget API? Any information is helpful. Lastly, >> does anyone have any hacky ideas of how to bury the source code so that the >> widgets merely display, rather than the user copy/pasting the widget source >> into a cell? >> >> Thanks From doug.blank at gmail.com Sat Aug 23 13:02:01 2014 From: doug.blank at gmail.com (Doug Blank) Date: Sat, 23 Aug 2014 13:02:01 -0400 Subject: [IPython-dev] Drag image onto notebook Message-ID: Devs, It used to be the case (early IPython 2, I believe) that the attached code would allow an image to be embedded into a notebook by dragging and dropping it onto the notebook. However, now there is a message in the browser console: HTML Sanitizer img.src removed -> Object Is there a way to allow this to work? Also, is there another option to allow local images to be embedded in a notebook (and won't vanish if one re-executes a cell)? Thanks for advice, -Doug -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: drag-and-drop.js Type: application/x-javascript Size: 5833 bytes Desc: not available URL: From doug.blank at gmail.com Sat Aug 23 13:26:01 2014 From: doug.blank at gmail.com (Doug Blank) Date: Sat, 23 Aug 2014 13:26:01 -0400 Subject: [IPython-dev] IPython3/Jupyter Magics for Python-based kernels In-Reply-To: References: Message-ID: On Sat, Aug 23, 2014 at 1:12 AM, MinRK wrote: > I think if you want to add something like magics for Python-wrapped > kernels based on your magickernel.py code, that would be interesting. > Standardizing magics across kernel languages is something that is > attractive on its face, but we deemed outside the scope of the message / > kernel specification, because it is probably not feasible to do in a > language agnostic way (For example, IPython magics can contain Python > string-formatting code with Python expressions that are evaluated in the > interactive namespace). Clarification of discussions like this are part of > the reasoning behind the Jupyter/IPython split: > > - Jupyter is language-agnostic computing tools, such as the message > protocol, notebook document format, and notebook UI. > - IPython is a better interactive Python environment, with some > extensions to the Python language for interactive use. > - IPython provides a Jupyter Kernel. > - Magics are 100% an IPython feature, and not in any way related to > Jupyter. > - Other kernels are welcome to implement extensions to their language > (including Python!) as they see fit, > as similar or different to IPython?s magics system as makes sense. > > Implementing magics for basic Python-wrapped kernels (i.e. not R, Julia, > Ruby, Haskell, etc.) would be simple, and potentially useful, depending on > how many of these kernels are developed. I do not expect most first-class > kernels to go this route, in which case they would not share any magic > code. Further, I don?t think the magic code developed for this use case > would even be shared by IPython. > > But it would still be an interesting exercise, and might encourage other > kernels to take the multi-interpreter approach if it is successful. > I see your point, but from a general Jupyter user's perspective, IF a kernel supports a magic-like metacommand, THEN it would be nice to be uniform in syntax, and behavior. Imagine that 5 different kernels implement a magic like "%%time" with 5 different syntaxes, and 5 slightly different results. I am not advocating that the IPython team need to do anything. I am just suggesting that where possible, it would be great from the user's perspective if there was some standardization across these metacommands. I am finding that there are many "magics" (I know that I'm defining that word differently than IPython uses it) that can be implemented in a manner that doesn't affect (nor interact with) the underlying language. In MagicKernel, these commands act as pre and post processors, but are not "in" the language as they are in IPython's Python. Sounds like a fun experiment! > Well, it is very useful to have the set that are independent of the language, and we are implementing them. If they are useful for other Python-based kernels, then great! All of the following kernels in pip now have these standard magics: eval_kernel (simple Python evaluator), echo_kernel (from docs), chatbot_kernel, and scheme_kernel. -Doug > -MinRK > ? > > > On Wed, Aug 20, 2014 at 4:22 AM, Doug Blank wrote: > >> IPython developers, >> >> One of the nice things that the IPython team did for the next version was >> to create a base class for creating new kernels for code written in Python, >> called wrapper kernels [1]. >> >> Unfortunately, such kernels don't have ! shell commands, nor magics. This >> is understandable, as those are tightly integrated with IPython's Python. >> However, one could get some of the basic functionality of these features on >> top of the wrapper kernel, in a generic way. For example, you could run >> shell commands, or have some magics that operate independent of the kernel. >> >> We have created a prototype of magics for wrapper kernels [2]. The code >> is in magickernel.py, and the magics are in the folder magics. The current >> prototype works like this: >> >> ``` >> In [1]: %magics >> Line magics: >> %cd PATH - change current directory of session >> %connect_info - show connection information >> %download URL [FILENAME] - download file from URL >> %html CODE - display code as HTML >> %install_magic URL - download and install magic from URL >> %javascript CODE - send code as JavaScript >> %latex TEXT - display text as LaTex >> %magics - show installed magics >> %reload_magics - reload the magics from the installed files >> %shell COMMAND - run the line as a shell command >> %time COMMAND - show time to run line >> >> Cell magics: >> %%file FILENAME - write contents of cell to file >> %%html - display contents of cell as HTML >> %%javascript - send contents of cell as JavaScript >> %%latex - display contents of cell as LaTeX >> %%shell - run the contents of the cell as shell commands >> %%time - show time to run cell >> >> Shell shortcut: >> ! COMMAND ... - execute command in shell >> >> Any cell magic can be made persistent for rest of session by using %%% >> prefix. >> ``` >> >> The idea is to be able to easily add some standard-like magics to >> 3rd-party kernels by using MagicKernel. See the sample wrapperkernel >> EchoKernel with magics [3]. Also, the idea here is not to replicate too >> much what comes with IPython's Python, but to provide a small, simple core >> set of magics that can't be easily done in the kernel language. You can >> install MagicKernel from source, or using: >> >> $ pip install calico --upgrade >> >> This is a prototype (with some bugs), and we very much hope for others to >> provide feedback or code, and more work needs to be done. What magics would >> you need in your 3rd-party kernel, that can be accomplished this way? Will >> this design be sufficient? >> >> We hope that this is useful! >> >> -Doug >> >> [1] - http://ipython.org/ipython-doc/dev/development/wrapperkernels.html >> [2] - https://bitbucket.org/ipre/calico/src/master/src/calico/ >> [3] - >> https://bitbucket.org/ipre/calico/src/master/src/echo_kernel/echo_kernel.py >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick.bollweg at gmail.com Sat Aug 23 13:52:55 2014 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Sat, 23 Aug 2014 13:52:55 -0400 Subject: [IPython-dev] eco: a composed language editor Message-ID: More of a thought exercise than a discussion of actual features: i recently read about this editor for composed languages, eco: - site: http://soft-dev.org/src/eco/ - paper: http://tratt.net/laurie/blog/entries/an_editor_for_composed_programs - repo: https://bitbucket.org/softdevteam/eco/ Just from their examples, I now have this picture in my mind of a codemirror-based, eco-like editor in a cell that lets me prototype an interactive GUI driven by a couple SPARQL queries with python interpolated inside the querie, without any templating stuff in sight. I can mix in Widgets wherever I like. Hell, throw in some coffeescript and LESS. At first glance, it's a 90's PHP nightmare, but then I remember that I can immediately tell if all of the layers are syntactically correct, since each layer was interactively built against a grammar, augmented by what is known about the current running environment. Some things I thought about: - what is a composed kernel(s)? - they don't really cover how one might actually *execute *one of these files. for their example with HTML, Python, SQL, what's really happening? in my mind, this is really just a python kernel that wraps HTML and SQL as strings, but a more involved example? - how would parser-driven editing in context work? - the completion available in a running kernel is already far beyond what a grammar-only completion tool can do. what would a more aggressively parser-driven editing experience be like? - how would one store a tree of tokens? - presumably, storing serialized JSON in cells[].source, while ugly, would work. or metadata. - what are visual tokens like? - actually, in line with the jupyter iconography (i.e. galileo's moon), the inlining of visual tokens would be incredible. codemirror doesn't appear to support flowed widgets (yet?). their suggested "real" use, of a formula editor, is interesting. And, of course, matrices. - eco native format is gzipped JSON - i think this has been discussed before (ipynbz). i don't know what advantage this provides over just having the json out there, but i have created some pretty big notebooks before. also, notebook files are not really hand editable (though I have done so). Interested in what other folks think... Cheers! Nick -------------- next part -------------- An HTML attachment was scrubbed... URL: From hughesadam87 at gmail.com Sat Aug 23 13:49:59 2014 From: hughesadam87 at gmail.com (hugadams) Date: Sat, 23 Aug 2014 10:49:59 -0700 (PDT) Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: Wow, thanks for sharing. This is going to be an excellent resource! Since this thread probably has the most widget experts I'm ever going to find, may I ask a followup that I was chewing over on a plane ride yesterday? For our package, I think it would be beneficial to make a series of useful containers of various widgets for a particular task. For example, a "filtering" container would have a series of widgets related to filtering data. Maybe a dropdown widget with "fourier, hamming, wavelet" and other related things like a "transform" and "back transform button". Likewise, maybe a 1-d plotting widget would store a plot, have a save button, publicize some common plotting options like "color" and so forth. I'd like to build these, and then assemble them into more complex GUI's, where the main gui would define all of the events to link up all of these sub-UI's, or conatiners as I'm calling them for lack of a better word. What's unclear to me is if a complex event system like this is feasible in the current design of the widget API. Let's say I have two sub-UI's for Plotting and Filtering. Most likely, they'll be in some sort of class, so pretend I could instantiate as: plot_ui = Plotting() filtering_ui = Filtering() Imagine a simple event where the user changes an attribute through a widget in the filtering UI and this is to trigger a redraw of the plot. In my main program, I'd like to link these up. Is this type of functionality supported? Something like: @on_trait_change(filtering_ui.windowtype): if filtering_ui.windowtype == 'hamming': plot_ui.redraw(scipy.filter.hamming(data) etc... On Sat, Aug 23, 2014 at 12:45 PM, rossant [via Python] < ml-node+s6n5069009h41 at n6.nabble.com> wrote: > Adam, > > FYI, you'll find my EuroSciPy tutorial on the widgets here > > > Best, > Cyrille > > 2014-08-20 10:15 GMT+02:00 Cyrille Rossant <[hidden email] > >: > > > Hi Adam, > > > > I'll do a tutorial about IPython widgets at EuroSciPy in a week [1]. > > I'm notably planning to demonstrate how to build, in the notebook, a > > GUI like the one you showed. I'll put here the link to the GitHub repo > > when everything's ready. > > > > Best, > > Cyrille > > > > [1] https://www.euroscipy.org/2014/schedule/presentation/60/ > > > > 2014-08-17 19:54 GMT+02:00 hugadams <[hidden email] > >: > >> (Please disregard if doublepost; having some subscription issues) > >> > >> Hello, > >> > >> We are developing a spectroscopy library and want to put in some > interactive > >> plots/utilities to make the software a bit more approachable to > novices. > >> I'd really like to build the entire interactive plotting interface > through > >> the notebook's new widget system. My reasoning is as follows: > >> > >> - All static plots are already matplotlib-based, so we wouldn't have > to > >> write a seperate plotting api > >> - Notebooks are inherently browser-ready, so sharing plots would be > great > >> - Is it likely that static nbviewer will eventually support > >> widgets? Since the data is required, I figured not... > >> - Our tutorials are already in notebooks, so we'd like to be > consistent. > >> > >> I've watched all the videos I can find from the widgets, and think I'm > fine > >> with the Python end; however, am finding it hard to come by lots of > >> information on controlling the layout of the widgets. I know it's > mostly > >> done in javascript, and I think the most complex view we will have will > look > >> something like this: > >> > >> https://sites.google.com/site/shigemorita/home/2dshige > >> > >> How much time would you estimate a javascript novice ought to spend on > >> getting something like that in the notebook? Can anyone share > tutorials for > >> the layout portion of the widget API? Any information is helpful. > Lastly, > >> does anyone have any hacky ideas of how to bury the source code so that > the > >> widgets merely display, rather than the user copy/pasting the widget > source > >> into a cell? > >> > >> Thanks > _______________________________________________ > IPython-dev mailing list > [hidden email] > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069009.html > To unsubscribe from Feasilibity of widgets/layout for mimicking this, click > here > > . > NAML > > -- View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069013.html Sent from the IPython - Development mailing list archive at Nabble.com. -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at elbonia.de Sat Aug 23 14:06:42 2014 From: python at elbonia.de (Juergen Hasch) Date: Sat, 23 Aug 2014 20:06:42 +0200 Subject: [IPython-dev] Drag image onto notebook In-Reply-To: References: Message-ID: <53F8D832.6090906@elbonia.de> This is not possible due to the HTML sanitizer, as you noticed. Google-caja sanitizes base64 uris. There is an updated extension on https://github.com/ipython-contrib/IPython-notebook-extensions using the new 3.x IPython contents webservice. Dragging an image into a notebook now uploads it to the server. Am 23.08.2014 19:02, schrieb Doug Blank: > Devs, > > It used to be the case (early IPython 2, I believe) that the attached code would allow an image to be embedded into a > notebook by dragging and dropping it onto the notebook. > > However, now there is a message in the browser console: > > HTML Sanitizer img.src removed -> Object > > Is there a way to allow this to work? > > Also, is there another option to allow local images to be embedded in a notebook (and won't vanish if one re-executes a > cell)? > > Thanks for advice, > > -Doug > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From jason-sage at creativetrax.com Sat Aug 23 15:33:46 2014 From: jason-sage at creativetrax.com (Jason Grout) Date: Sat, 23 Aug 2014 15:33:46 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: <53F8EC9A.7090003@creativetrax.com> On 8/23/14, 13:49, hugadams wrote: > What's unclear to me is if a complex event system like this is feasible > in the current design of the widget API. Yes. Resoundingly yes. Someone (you?) should go for it---it would be pretty straightforward with the current design, and would be a great thing to see. Thanks, Jason From nick.bollweg at gmail.com Sat Aug 23 15:50:25 2014 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Sat, 23 Aug 2014 15:50:25 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: One way to do this would be to instead of requiring the notebook user to understand everything, expose important bits as traitlets, and then change/traitlet.link all of them together. In your example, if: - Plotting knows to redraw itself when its data changes - Filtering , updates its windowtype, to be, instead of "hamming", scipy.filter.hamming then suddenly your code is now: > def change_windowtype(name, old_value, new_value): > plot_ui.data = filtering_ui.windowtype(data) > filtering_ui.on_trait_change("windowtype", change_windowtype) > (I don't think your usage of on_trait_changed as a decorator is legit, but that would be awesome!) That moves a lot of the event stuff into the internal structure of each of your GUI components. This pattern would also be much easier to wrap up later into a "higher order" GUI. Then, if you want to "hoist" a property, you would just use traitlet.link: > class GUI(widgets.ContainerWidget) > windowtype = traitlets.Any(sync=True) > plot_data = traitlets.Any(sync=True) > data = traitlets.Any(sync=True) > > def __init__(self, **kwargs) > super(GUI, self).__init__(**kwargs) > > self.filtering_ui = Filtering() > self.plot_ui = Plotting() > > self.children = [self.filtering_ui, self.plot_ui] > > traitlets.link(self, "plot_data"), (self.plot_ui, "data"), > traitlets.link(self, "windowtype"), (self.filtering_ui, > "windowtype"), > > def _windowtype_changed(self, name, old_value, new_value): > self.plot_data = self.windowtype(self.data) > And now the notebook user would still be able to inspect and change the important parts of even deeply nested settings with simple assignment. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paddy at paddymullen.com Mon Aug 25 08:45:42 2014 From: paddy at paddymullen.com (Paddy Mullen) Date: Mon, 25 Aug 2014 08:45:42 -0400 Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter In-Reply-To: References: Message-ID: Would it make sense to give notebooks a preamble cell where the notebook could check that all of its required dependencies are installed. I guess this could be done by convention. Notebooks could also include a dependency list in requirements.txt or conda.yaml or a docker file, but the installation of those venvs is left as an implementation detail. On Fri, Aug 22, 2014 at 2:40 PM, Filipe Pires Alvarenga Fernandes < ocefpaf at gmail.com> wrote: > On Fri, Aug 22, 2014 at 2:34 PM, Fernando Perez > wrote: > >> We looked into it and as far as we can tell, all the pieces we're >> providing for the kernel spec should make it possible to have your kernel >> entry point do the switching. >> > > OK, I will keep trying to have something more robust. Right now I am > doing a very hackish workaround. > > >> But we will NOT bake into the kernel spec direct knowledge of venv, conda >> envs, or any other 'enviornment manager'. We want to make it *possible* do >> declare kernels that are connected to environments, but as of yet, there >> are just too many ways of 'managing environments' across languages and >> toolchains for us to enshrine any particular approach into the spec itself. >> > > I understand (and I agree) with that. > > >> As I say above, we looked into it, but we didn't actually implement >> anything yet. So we'd encourage you to try things out, and report both with >> what works (I'm sure others are interested) and what doesn't (so we can >> help out or fix things if needed) >> > > As soon as I have something better I will post it here. > > >> Cheers >> >> f >> > > Thanks, > > -Filipe > > >> >> >> On Fri, Aug 22, 2014 at 8:55 AM, Filipe Pires Alvarenga Fernandes < >> ocefpaf at gmail.com> wrote: >> >>> Have anyone tried to use venvs as kernels? Making the installed vens >>> show up in the drop-down menu in the new Jupyter Notebook? >>> >>> I was toying with the idea but got nowhere near something useful. >>> >>> I think it would be a nice way to switch between virtual environments. >>> >>> -Filipe >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://mail.scipy.org/mailman/listinfo/ipython-dev >>> >>> >> >> >> -- >> Fernando Perez (@fperez_org; http://fperez.org) >> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) >> fernando.perez-at-berkeley: contact me here for any direct mail >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Aug 25 08:55:59 2014 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Mon, 25 Aug 2014 14:55:59 +0200 Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter In-Reply-To: References: Message-ID: <8B05D1E8-DB46-423A-A486-911C2AF105F7@gmail.com> Le 25 ao?t 2014 ? 14:45, Paddy Mullen a ?crit : > Would it make sense to give notebooks a preamble cell where the notebook could check that all of its required dependencies are installed. I guess this could be done by convention. Notebooks could also include a dependency list in requirements.txt or conda.yaml or a docker file, but the installation of those venvs is left as an implementation detail. This as already been discusses here and there on the mailing list, and we had decided that this is outside of IPython scope. Konrad Hinsen have done Active Paper which is meant to improve things like that. Everything is an implementation detail ! -- M From benjaminrk at gmail.com Mon Aug 25 12:40:36 2014 From: benjaminrk at gmail.com (MinRK) Date: Mon, 25 Aug 2014 09:40:36 -0700 Subject: [IPython-dev] Using virtualenvs as kernels for Jupyter In-Reply-To: <8B05D1E8-DB46-423A-A486-911C2AF105F7@gmail.com> References: <8B05D1E8-DB46-423A-A486-911C2AF105F7@gmail.com> Message-ID: Aron Ahmadia has an extension (thisworks ) that ensures dependencies are installed, and if not, creates a new hashdist and relaunches a notebook server in the environment with all of the dependencies installed. ? On Mon, Aug 25, 2014 at 5:55 AM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > > Le 25 ao?t 2014 ? 14:45, Paddy Mullen a ?crit : > > > Would it make sense to give notebooks a preamble cell where the notebook > could check that all of its required dependencies are installed. I guess > this could be done by convention. Notebooks could also include a > dependency list in requirements.txt or conda.yaml or a docker file, but the > installation of those venvs is left as an implementation detail. > > This as already been discusses here and there on the mailing list, > and we had decided that this is outside of IPython scope. > > Konrad Hinsen have done Active Paper which is meant to improve things like > that. > > Everything is an implementation detail ! > -- > M > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Aug 26 04:53:45 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 26 Aug 2014 09:53:45 +0100 Subject: [IPython-dev] nbopen & nbmanager - a pair of tools for IPython Notebook on your own machine In-Reply-To: References: Message-ID: On 22 August 2014 07:48, Joshua Barratt wrote: > It *almost* works on IPython 2.x, except the nbserver-* JSON blobs only > have the PID's in the name, not the actual file. > > I took a pass at fixing this up for IPython 2.x, by borrowing some of the > code from the master branch, and it seems to be working. Great! I was just being lazy with my own implementation, because I live on the development version anyway. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Tue Aug 26 10:49:05 2014 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 26 Aug 2014 10:49:05 -0400 Subject: [IPython-dev] Listing books on the home page, a policy proposal. Feedback? In-Reply-To: References: Message-ID: +1 On Thu, Aug 21, 2014 at 4:35 PM, Fernando Perez wrote: > Hi folks, > > we just went over this on the dev meeting, but I'd like to give it a day or > two here on-list for feedback in case anyone who wasn't at the meeting has > further comments... > > For a while we've had on the homepage a link to Cyrille's IPython book, and > we've been contacted by a publisher of an upcoming title asking for a new > listing there. So I'd like to have uniform criteria that we use to make that > decision in a fair manner with *any* publisher. Here's what we discussed > during the meeting, with some additional refinements I just thought of to > make things clearer. > > One quick note: Brian noted that the website is way too busy already, which > is true. But I want to decouple this specific decision about books from an > entire website redesign conversation. I think that books are one of the > things that actually should belong on the main homepage, as they are a very > important resource related to the project and that benefits it in multiple > ways. So let's assume that any redesigned home page will have *some* space > for a small image pointing to a book, and the discussion here is about what > to put there and how to provide a policy publishers can all know about. > > Any feedback welcome. > > Cheers > > f > > #### Proposed language > > Our policy for listing books on the website is as follows. This will be > uniformly applied across publishers, so the project can maintain its > neutrality regarding IPython-related books (all language below would apply > in the future to Jupyter-oriented books). > > 1. Criteria for inclusion in the homepage > > We will list books that are specifically and mostly about IPython itself, > rather than just mentioning IPython in a chapter or two (though see Note in > #2 below). > > For example, the currently listed book by Cyrille Rossant amply qualifies, > but Wes McKinney's "Python for Data Analysis", while having an excellent > chapter devoted to IPython, would not get a link on the home page. > > > 2. Our "Books" page: > > We will create a main page listing in reverse chronological order the > various books that are mainly devoted to IPython. For each book, we simply > list its title, author and actual publisher page (or Amazon, whatever the > author prefers), along with a cover image. No editorializing/reviews/etc. > > There may be a section at the bottom listing books that include some useful > material about IPython (more than just a sentence but less than say 1/2 the > book). These would be listed for reference but would have no front-page > link (e.g. Wes' book). > > Note: the core team reserves the right to *not* include a book in this > listing. If someone out there publishes a book that we deem to be absolutely > terrible, we reserve the right to simply not list it. > > > 3. What we show on the homepage > > We will have a single image for books, and that will be a randomly rotating > screenshot of all currently available books. That image will be a link to > that book's entry in the "Books" page. > > Since now we'll only have at most two of these (the current one and > Cyrille's upcoming one), we can make the distribution give say 2/3 of views > to the new book and 1/3 to the old. If eventually there's multiple books, we > can figure out a *simple* distribution to highlight newer ones more often > while still showing the older ones (as long as they are in print). > > -- > Fernando Perez (@fperez_org; http://fperez.org) > fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) > fernando.perez-at-berkeley: contact me here for any direct mail > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo @ellisonbg on Twitter and GitHub bgranger at calpoly.edu and ellisonbg at gmail.com From david.trem at gmail.com Tue Aug 26 15:15:23 2014 From: david.trem at gmail.com (=?ISO-8859-1?Q?Tr=E9mouilles_David?=) Date: Tue, 26 Aug 2014 21:15:23 +0200 Subject: [IPython-dev] ipython notebook "current cell changed signal" Message-ID: <53FCDCCB.3090208@gmail.com> Hello, In the IPython notebook, I would like to trigger a javascript function when the notebook currently selected cell changes. I have found this kind of solution: %%javascript $([IPython.events]).on('select.Cell', function (event, data) { console.log("changed...") }); However it only works when the cell is changed by clicking on another one but not when navigating with the up and down arrow key... Is there any simple solution to trigger a javascript function whenever the selected cell changes ? (I'm using the current ipython github repo head.) Thanks in advance, David From tritemio at gmail.com Tue Aug 26 19:29:22 2014 From: tritemio at gmail.com (Antonino Ingargiola) Date: Tue, 26 Aug 2014 16:29:22 -0700 Subject: [IPython-dev] ipython notebook widgets: combobox? Message-ID: Hi to the list, I'm playing with the widgets and I can't find any example of a combo-box widget. I need a numeric text entry (like BoundedIntTextWidget) with two small buttons to increment or decrement the value. All the GUI toolkits have something similar. If this widget does not yet exist, would it be possible to bind two buttons (Up, Down) to a BoundedIntTextWidget in order to achieve roughly the same result? Thanks, Antonio -------------- next part -------------- An HTML attachment was scrubbed... URL: From cyrille.rossant at gmail.com Tue Aug 26 19:46:04 2014 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Wed, 27 Aug 2014 00:46:04 +0100 Subject: [IPython-dev] ipython notebook widgets: combobox? In-Reply-To: References: Message-ID: Hi See example 4 here: https://github.com/rossant/euroscipy2014 Cyrille Le mercredi 27 ao?t 2014, Antonino Ingargiola a ?crit : > Hi to the list, > > I'm playing with the widgets and I can't find any example of a combo-box > widget. I need a numeric text entry (like BoundedIntTextWidget) with two > small buttons to increment or decrement the value. All the GUI toolkits > have something similar. > > If this widget does not yet exist, would it be possible to bind two > buttons (Up, Down) to a BoundedIntTextWidget in order to achieve roughly > the same result? > > Thanks, > Antonio > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.trem at gmail.com Wed Aug 27 05:17:37 2014 From: david.trem at gmail.com (=?ISO-8859-1?Q?Tr=E9mouilles_David?=) Date: Wed, 27 Aug 2014 11:17:37 +0200 Subject: [IPython-dev] ipython notebook "current cell changed signal" In-Reply-To: <53FCDCCB.3090208@gmail.com> References: <53FCDCCB.3090208@gmail.com> Message-ID: <53FDA231.5080607@gmail.com> OK. I have finally figured this out by myself :-) 'selected_cell_type_changed.Notebook' is actually what I was looking for. I was misled by the naming. My first thought was that the 'selected_cell_type_changed.Notebook' signal was /only/ triggered when the selected "cell type" was changed, actually it is triggered either when the "selected cell" or the "selected cell type" are changed... Best regards, David Le 26/08/14 21:15, Tr?mouilles David a ?crit : > Hello, > > In the IPython notebook, I would like to trigger a javascript > function when the notebook currently selected cell changes. > > I have found this kind of solution: > > %%javascript > $([IPython.events]).on('select.Cell', function (event, data) { > console.log("changed...") > }); > > However it only works when the cell is changed by clicking on > another one but not when navigating with the up and down arrow key... > Is there any simple solution to trigger a javascript function > whenever the selected cell changes ? > > (I'm using the current ipython github repo head.) > > Thanks in advance, > > David -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Aug 27 05:34:04 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 27 Aug 2014 10:34:04 +0100 Subject: [IPython-dev] ipython notebook "current cell changed signal" In-Reply-To: <53FDA231.5080607@gmail.com> References: <53FCDCCB.3090208@gmail.com> <53FDA231.5080607@gmail.com> Message-ID: That could be a bug - if the events don't make sense at present, be aware that we may fix them in a future release. On 27 August 2014 10:17, Tr?mouilles David wrote: > OK. I have finally figured this out by myself :-) > > 'selected_cell_type_changed.Notebook' is > actually what I was looking for. > > I was misled by the naming. My first thought was > that the 'selected_cell_type_changed.Notebook' signal > was *only* triggered when the selected "cell type" was > changed, actually it is triggered either when the > "selected cell" or the "selected cell type" are changed... > > Best regards, > > David > > Le 26/08/14 21:15, Tr?mouilles David a ?crit : > > Hello, > > In the IPython notebook, I would like to trigger a javascript > function when the notebook currently selected cell changes. > > I have found this kind of solution: > > %%javascript > $([IPython.events]).on('select.Cell', function (event, data) { > console.log("changed...") > }); > > However it only works when the cell is changed by clicking on > another one but not when navigating with the up and down arrow key... > Is there any simple solution to trigger a javascript function > whenever the selected cell changes ? > > (I'm using the current ipython github repo head.) > > Thanks in advance, > > David > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.trem at gmail.com Wed Aug 27 07:04:26 2014 From: david.trem at gmail.com (=?ISO-8859-1?Q?Tr=E9mouilles_David?=) Date: Wed, 27 Aug 2014 13:04:26 +0200 Subject: [IPython-dev] ipython notebook "current cell changed signal" In-Reply-To: References: <53FCDCCB.3090208@gmail.com> <53FDA231.5080607@gmail.com> Message-ID: <53FDBB3A.4080005@gmail.com> Yes indeed, this might be considered as a bug. I'll let you (core developers) decide whether or not... Is there any plan to propose and document a kind of notebook javascript API ? (I did not find such document in the IPython docs but it could be that I was not enough persistent (?)) Anyway, thanks to all of you for IPython. It rocks! David Le 27/08/14 11:34, Thomas Kluyver a ?crit : > That could be a bug - if the events don't make sense at present, be > aware that we may fix them in a future release. > > > On 27 August 2014 10:17, Tr?mouilles David > wrote: > > OK. I have finally figured this out by myself :-) > > 'selected_cell_type_changed.Notebook' is > actually what I was looking for. > > I was misled by the naming. My first thought was > that the 'selected_cell_type_changed.Notebook' signal > was /only/ triggered when the selected "cell type" was > changed, actually it is triggered either when the > "selected cell" or the "selected cell type" are changed... > > Best regards, > > David > > Le 26/08/14 21:15, Tr?mouilles David a ?crit : >> Hello, >> >> In the IPython notebook, I would like to trigger a javascript >> function when the notebook currently selected cell changes. >> >> I have found this kind of solution: >> >> %%javascript >> $([IPython.events]).on('select.Cell', function (event, data) { >> console.log("changed...") >> }); >> >> However it only works when the cell is changed by clicking on >> another one but not when navigating with the up and down arrow >> key... >> Is there any simple solution to trigger a javascript function >> whenever the selected cell changes ? >> >> (I'm using the current ipython github repo head.) >> >> Thanks in advance, >> >> David > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Aug 27 07:11:57 2014 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 27 Aug 2014 12:11:57 +0100 Subject: [IPython-dev] ipython notebook "current cell changed signal" In-Reply-To: <53FDBB3A.4080005@gmail.com> References: <53FCDCCB.3090208@gmail.com> <53FDA231.5080607@gmail.com> <53FDBB3A.4080005@gmail.com> Message-ID: On 27 August 2014 12:04, Tr?mouilles David wrote: > Is there any plan to propose and document > a kind of notebook javascript API ? (I did not > find such document in the IPython docs but > it could be that I was not enough persistent (?)) > That is the plan, but there's nothing to find yet. The JS API has been changing quite quickly, but when it's settled down enough, we do want to define and document what people should rely on. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Wed Aug 27 07:15:37 2014 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Wed, 27 Aug 2014 12:15:37 +0100 Subject: [IPython-dev] ipython notebook "current cell changed signal" In-Reply-To: References: <53FCDCCB.3090208@gmail.com> <53FDA231.5080607@gmail.com> <53FDBB3A.4080005@gmail.com> Message-ID: <47404C39-C66F-4E9B-B981-A77B73D69801@gmail.com> Le 27 ao?t 2014 ? 12:11, Thomas Kluyver a ?crit : > On 27 August 2014 12:04, Tr?mouilles David wrote: > Is there any plan to propose and document > a kind of notebook javascript API ? (I did not > find such document in the IPython docs but > it could be that I was not enough persistent (?)) > > That is the plan, but there's nothing to find yet. The JS API has been changing quite quickly, but when it's settled down enough, we do want to define and document what people should rely on. And the more people we have to **maintain**[1] javascript/HTML and write doc, the sooner this will happen. ? M PS.: did that some time ago: http://ipjsdoc.herokuapp.com/ [1]: that does not mean add new features :-) > > Thomas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug.blank at gmail.com Wed Aug 27 12:24:11 2014 From: doug.blank at gmail.com (Doug Blank) Date: Wed, 27 Aug 2014 12:24:11 -0400 Subject: [IPython-dev] What's the easiest way to render an .ipynb file as html? Message-ID: Short of calling nbviewer, what is the easiest way to render a local .ipynb file as html, just like nbviewer does? Would prefer to use the IPython libraries directly rather than externally calling ipython, but that would work too. -Doug -------------- next part -------------- An HTML attachment was scrubbed... URL: From jakevdp at cs.washington.edu Wed Aug 27 12:54:11 2014 From: jakevdp at cs.washington.edu (Jacob Vanderplas) Date: Wed, 27 Aug 2014 09:54:11 -0700 Subject: [IPython-dev] What's the easiest way to render an .ipynb file as html? In-Reply-To: References: Message-ID: Here's an example of a possible approach: from IPython.nbformat import current as nbformat from IPython.nbconvert.exporters import HTMLExporter input_file = 'notebook.ipynb' output_file = 'notebook.html' with open(input_file) as f: notebook_content = f.read() exporter = HTMLExporter(template_file='full') nb_json = nbformat.reads_json(notebook_content) (body, resources) = exporter.from_notebook_node(nb_json) with open(output_file, 'w') as f: f.write(body) Hope that helps! Jake Jake VanderPlas Director of Research ? Physical Sciences eScience Institute, University of Washington http://www.vanderplas.com On Wed, Aug 27, 2014 at 9:24 AM, Doug Blank wrote: > Short of calling nbviewer, what is the easiest way to render a local > .ipynb file as html, just like nbviewer does? Would prefer to use the > IPython libraries directly rather than externally calling ipython, but that > would work too. > > -Doug > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug.blank at gmail.com Wed Aug 27 13:25:45 2014 From: doug.blank at gmail.com (Doug Blank) Date: Wed, 27 Aug 2014 13:25:45 -0400 Subject: [IPython-dev] What's the easiest way to render an .ipynb file as html? In-Reply-To: References: Message-ID: Great, thanks! That gets me very close to being complete... need to make sure I get the css to load. (I'm adding a method to dynamically create a static version of a live, shared notebook in the new multiuser server [1]). -Doug [1] - https://github.com/jupyter/jupyterhub On Wed, Aug 27, 2014 at 12:54 PM, Jacob Vanderplas < jakevdp at cs.washington.edu> wrote: > Here's an example of a possible approach: > > from IPython.nbformat import current as nbformat > from IPython.nbconvert.exporters import HTMLExporter > > input_file = 'notebook.ipynb' > output_file = 'notebook.html' > > with open(input_file) as f: > notebook_content = f.read() > > exporter = HTMLExporter(template_file='full') > nb_json = nbformat.reads_json(notebook_content) > (body, resources) = exporter.from_notebook_node(nb_json) > > with open(output_file, 'w') as f: > f.write(body) > > Hope that helps! > Jake > > Jake VanderPlas > Director of Research ? Physical Sciences > eScience Institute, University of Washington > http://www.vanderplas.com > > > On Wed, Aug 27, 2014 at 9:24 AM, Doug Blank wrote: > >> Short of calling nbviewer, what is the easiest way to render a local >> .ipynb file as html, just like nbviewer does? Would prefer to use the >> IPython libraries directly rather than externally calling ipython, but that >> would work too. >> >> -Doug >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hughesadam87 at gmail.com Wed Aug 27 13:33:24 2014 From: hughesadam87 at gmail.com (Adam Hughes) Date: Wed, 27 Aug 2014 13:33:24 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: Thanks for the help Nick and Jason, sorry for the late reply. WIll try out your suggestions. PS, the trait notification decorator is standard to Traits but probably not Traitlets. On Sat, Aug 23, 2014 at 3:50 PM, Nicholas Bollweg wrote: > One way to do this would be to instead of requiring the notebook user to > understand everything, expose important bits as traitlets, and then > change/traitlet.link all of them together. In your example, if: > > - Plotting knows to redraw itself when its data changes > - Filtering , updates its windowtype, to be, instead of "hamming", > scipy.filter.hamming > > then suddenly your code is now: > >> def change_windowtype(name, old_value, new_value): >> plot_ui.data = filtering_ui.windowtype(data) >> filtering_ui.on_trait_change("windowtype", change_windowtype) >> > (I don't think your usage of on_trait_changed as a decorator is legit, but > that would be awesome!) > > That moves a lot of the event stuff into the internal structure of each of > your GUI components. This pattern would also be much easier to wrap up > later into a "higher order" GUI. Then, if you want to "hoist" a property, > you would just use traitlet.link: > >> class GUI(widgets.ContainerWidget) >> windowtype = traitlets.Any(sync=True) >> plot_data = traitlets.Any(sync=True) >> data = traitlets.Any(sync=True) >> >> def __init__(self, **kwargs) >> super(GUI, self).__init__(**kwargs) >> >> self.filtering_ui = Filtering() >> self.plot_ui = Plotting() >> >> self.children = [self.filtering_ui, self.plot_ui] >> >> traitlets.link(self, "plot_data"), (self.plot_ui, "data"), >> traitlets.link(self, "windowtype"), (self.filtering_ui, >> "windowtype"), >> >> def _windowtype_changed(self, name, old_value, new_value): >> self.plot_data = self.windowtype(self.data) >> > And now the notebook user would still be able to inspect and change the > important parts of even deeply nested settings with simple assignment. > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritemio at gmail.com Wed Aug 27 14:41:19 2014 From: tritemio at gmail.com (Antonino Ingargiola) Date: Wed, 27 Aug 2014 11:41:19 -0700 Subject: [IPython-dev] ipython notebook widgets: combobox? In-Reply-To: References: Message-ID: Thanks Cyrille, your collection of notebooks for euroscipy2014 is amazing! I let you know if I encounter problems in implementing the combo-box. Antonio On Tue, Aug 26, 2014 at 4:46 PM, Cyrille Rossant wrote: > Hi > See example 4 here: https://github.com/rossant/euroscipy2014 > Cyrille > > Le mercredi 27 ao?t 2014, Antonino Ingargiola a > ?crit : > > Hi to the list, >> >> I'm playing with the widgets and I can't find any example of a combo-box >> widget. I need a numeric text entry (like BoundedIntTextWidget) with two >> small buttons to increment or decrement the value. All the GUI toolkits >> have something similar. >> >> If this widget does not yet exist, would it be possible to bind two >> buttons (Up, Down) to a BoundedIntTextWidget in order to achieve roughly >> the same result? >> >> Thanks, >> Antonio >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Wed Aug 27 15:52:56 2014 From: matthew.brett at gmail.com (Matthew Brett) Date: Wed, 27 Aug 2014 15:52:56 -0400 Subject: [IPython-dev] javascript / Python communication for image viewer Message-ID: Guys / gals, I want to ask for advice about writing a brain image display widget for IPython. I would like to make an IPython widget that can take an in-memory numpy array and do an interactive display of orthogonal slices from the array. The display will look something like Papaya: http://rii.uthscsa.edu/mango/papaya where clicking or moving the mouse causes matching slices to be displayed through the three axes of the numpy array (here a brain image). Papaya is pure javascript, so I am assuming that it loads the whole array (brain image) into a javascript variable and takes slices from that. What I would like to do, is to be able to keep the whole 3D array only in Python, and pass the slices as needed to a javascript viewer. In my ignorance, I am not sure which approach to go for first. Should I use the comm / widget interface for this? In that case I guess the procedure would be: * mouse movement generates a 'need slice' message from javascript * python kernel accepts 'need slice' message, takes slice from array, base64 encodes into JSON, sends 'here is your slice' message back to javascript with the data * javascript accepts message, unpacks base64'ed JSONed slice into variable and displays slice variable Is that right? I guess that involves Is there any chance that this will be fast enough for satisfying interactive movement through the image, which would likely require something like 20 slices per second, each of say 64 x 64 floating point? If not - is there something else I should look at instead? Another question for more thanks - should I use a Canvas or SVG element to display the images? The fabric.js README [1] seems to imply the Canvas element is faster for some interactive stuff, does anyone have relevant experience to share? Thanks a lot, Matthew [1] https://github.com/kangax/fabric.js/#history From cyrille.rossant at gmail.com Wed Aug 27 16:53:28 2014 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Wed, 27 Aug 2014 21:53:28 +0100 Subject: [IPython-dev] javascript / Python communication for image viewer In-Reply-To: References: Message-ID: We have pretty similar requirements for Vispy, as we target fast visualization of big datasets using OpenGL in the IPython notebook. In particular, we can't use Canvas or SVG because it's just too slow for big data, so we have to use WebGL. This lets us leverage the GPU for visualization. I would be very interested in seeing the answers to your questions. Notably, is it possible to use binary sockets for transferring large amounts of binary data between Python and JavaScript? Cyrille 2014-08-27 20:52 GMT+01:00 Matthew Brett : > Guys / gals, > > I want to ask for advice about writing a brain image display widget for > IPython. > > I would like to make an IPython widget that can take an in-memory numpy array > and do an interactive display of orthogonal slices from the array. The > display will look something like Papaya: > > http://rii.uthscsa.edu/mango/papaya > > where clicking or moving the mouse causes matching slices to be displayed > through the three axes of the numpy array (here a brain image). > > Papaya is pure javascript, so I am assuming that it loads the whole array > (brain image) into a javascript variable and takes slices from that. > > What I would like to do, is to be able to keep the whole 3D array only in > Python, and pass the slices as needed to a javascript viewer. > > In my ignorance, I am not sure which approach to go for first. > > Should I use the comm / widget interface for this? In that case I guess the > procedure would be: > > * mouse movement generates a 'need slice' message from javascript > * python kernel accepts 'need slice' message, takes slice from array, base64 > encodes into JSON, sends 'here is your slice' message back to javascript > with the data > * javascript accepts message, unpacks base64'ed JSONed slice into variable and > displays slice variable > > Is that right? I guess that involves Is there any chance that this > will be fast enough for > satisfying interactive movement through the image, which would likely require > something like 20 slices per second, each of say 64 x 64 floating point? > > If not - is there something else I should look at instead? > > Another question for more thanks - should I use a Canvas or SVG element to > display the images? The fabric.js README [1] seems to imply the > Canvas element is > faster for some interactive stuff, does anyone have relevant experience to > share? > > Thanks a lot, > > Matthew > > [1] https://github.com/kangax/fabric.js/#history > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From nick.bollweg at gmail.com Wed Aug 27 18:29:47 2014 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Wed, 27 Aug 2014 18:29:47 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: Adam: It's really no problem... this is exactly the way we want to be building engineering tools at work, so it's been beneficial to have a more-defined problem than "make interactive dashboards better/faster/stronger". I really dug into the 3.0 notebook over the weekend, and do find it a bit nicer. Also, your decorator made me want some similar stuff, so I hacked together these two classes: http://nbviewer.ipython.org/gist/bollwyvl/8001ed62c6868301c26e > again, no widget actually show up, but I think one gets the idea. it does provide a decorator (I'm not happy with it yet) as well as overloaded assignment and transformed values. I hope this might be useful! -------------- next part -------------- An HTML attachment was scrubbed... URL: From hughesadam87 at gmail.com Wed Aug 27 19:47:10 2014 From: hughesadam87 at gmail.com (hugadams) Date: Wed, 27 Aug 2014 16:47:10 -0700 (PDT) Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: These are def cool. Can you keep me updated if you add more to this? I'd like to steal it (with credit) for our widgets. One question about the decorator: @wire(wire.e.value, wire.a.value)def thrice(a): return a * 3 Does this mean that anytime the "e" widget value is changed, the "a" widget value will be multiplied by 3? Or am I not understanding what is happening here? On Wed, Aug 27, 2014 at 6:24 PM, Nicholas Bollweg [via Python] < ml-node+s6n5069419h51 at n6.nabble.com> wrote: > Adam: > It's really no problem... this is exactly the way we want to be building > engineering tools at work, so it's been beneficial to have a more-defined > problem than "make interactive dashboards better/faster/stronger". > > I really dug into the 3.0 notebook over the weekend, and do find it a bit > nicer. > > Also, your decorator made me want some similar stuff, so I hacked together > these two classes: > > http://nbviewer.ipython.org/gist/bollwyvl/8001ed62c6868301c26e >> > > again, no widget actually show up, but I think one gets the idea. > > it does provide a decorator (I'm not happy with it yet) as well as > overloaded assignment and transformed values. I hope this might be useful! > > _______________________________________________ > IPython-dev mailing list > [hidden email] > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069419.html > To unsubscribe from Feasilibity of widgets/layout for mimicking this, click > here > > . > NAML > > -- View this message in context: http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069424.html Sent from the IPython - Development mailing list archive at Nabble.com. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick.bollweg at gmail.com Wed Aug 27 22:57:07 2014 From: nick.bollweg at gmail.com (Nicholas Bollweg) Date: Wed, 27 Aug 2014 22:57:07 -0400 Subject: [IPython-dev] Feasilibity of widgets/layout for mimicking this In-Reply-To: References: <1408298099961-5068197.post@n6.nabble.com> Message-ID: I was suggesting (with the naming) that e would be updated by a changing, but I agree, it's confusing. I tried something more complicated with kwargs, but it ended up being a whole second implementation of the thing that Wire does, which seemed like too much, but would look like this (which I think is somewhat more readable): @wire(wire.e.value, a=wire.a.value)def thrice(a): return a * 3 I've updated to use that, as well as an example of the dictionary-based assignment it is using. I welcome any other suggestions: the decorator syntax is somewhat limiting, which is probably good as this is already really evil :) On Wed, Aug 27, 2014 at 7:47 PM, hugadams wrote: > These are def cool. Can you keep me updated if you add more to this? I'd > like to steal it (with credit) for our widgets. > > One question about the decorator: > > @wire(wire.e.value, wire.a.value)def thrice(a): > return a * 3 > > > Does this mean that anytime the "e" widget value is changed, the "a" > widget value will be multiplied by 3? Or am I not understanding what is > happening here? > > On Wed, Aug 27, 2014 at 6:24 PM, Nicholas Bollweg [via Python] <[hidden > email] > wrote: > >> Adam: >> It's really no problem... this is exactly the way we want to be building >> engineering tools at work, so it's been beneficial to have a more-defined >> problem than "make interactive dashboards better/faster/stronger". >> >> I really dug into the 3.0 notebook over the weekend, and do find it a bit >> nicer. >> >> Also, your decorator made me want some similar stuff, so I hacked >> together these two classes: >> >> http://nbviewer.ipython.org/gist/bollwyvl/8001ed62c6868301c26e >>> >> >> again, no widget actually show up, but I think one gets the idea. >> >> it does provide a decorator (I'm not happy with it yet) as well as >> overloaded assignment and transformed values. I hope this might be useful! >> >> _______________________________________________ >> IPython-dev mailing list >> [hidden email] >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> >> ------------------------------ >> If you reply to this email, your message will be added to the >> discussion below: >> >> http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-tp5068197p5069419.html >> To unsubscribe from Feasilibity of widgets/layout for mimicking this, click >> here. >> NAML >> >> > > > ------------------------------ > View this message in context: Re: Feasilibity of widgets/layout for > mimicking this > > Sent from the IPython - Development mailing list archive > at > Nabble.com. > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jabooth at gmail.com Thu Aug 28 03:45:15 2014 From: jabooth at gmail.com (James Booth) Date: Thu, 28 Aug 2014 00:45:15 -0700 (PDT) Subject: [IPython-dev] javascript / Python communication for image viewer In-Reply-To: References: Message-ID: <1409211914670.efeba11e@Nodemailer> Hey Matthew, Cyrille, I?m developing a three.js-based web app for image/object annotation called landmarker.io (I work in computer vision/machine learning, getting high quality annotations of 3D and 2D data is often critical to training models). It?s located here www.landmarker.io (tool, launches in demo by default) https://github.com/menpo/landmarker.io (code) I?m also one of the developer?s of Menpo (www.menpo.io), which is a Python package for building and testing deformable models. I?m going to be developing an IPython widget-version of the landmarker.io tool, so we can correct annotations as we sport problems in the notebook. I mention all this as: 1. I?m also interested in understanding how we can transfer large arrays efficiently to IPython widgets 2. I have a little experience in how best to handle this in a traditional client/server model, but I don?t know how well this will translate to the comm interface. Basically, landmarker.io expects to be able to talk to a RESTful interface that is implemented currently by: https://github.com/menpo/landmarkerio-server Originally, I sent meshes as large JSON objects, but this was pretty slow as: 1. It?s more work in JS to parse the JSON into an array 2. It?s more work to turn this into the most efficient JS type for numerical arrays (which WebGL loves) which is an ArrayBuffer. https://developer.mozilla.org/en-US/docs/Web/API/ArrayBuffer https://developer.mozilla.org/en-US/docs/Web/API/Float32Array This meant that even with a server on localhost, skipping through faces to annotate felt a little sluggish. Instead now I just directly build a pure ArrayBuffer in Python on the server and ship it to the client. That happens here in the server: https://github.com/menpo/landmarkerio-server/blob/master/landmarkerio/cache.py#L165 (I?m a little lazy here and literally save the file to disk, then gzip it. It could be done in memory with StringIO through, but I do this as a caching process so it?s going to disk anyway). On the client side, I make an XMLHttpRequest?and demand an array buffer from the server: https://github.com/menpo/landmarker.io/blob/master/src/js/app/lib/get.js#L9 For completion, the actual parsing of the array buffer is done here: https://github.com/menpo/landmarker.io/blob/master/src/js/app/model/mesh.js#L104 Because three.js supports ArrayBuffers, this is really fast. I just point three to the array and we are away. With this implementation, browsing through subjects to annotate with a server on localhost (akin to moving between slices in your brain scan I imagine) is very fast. I?m afraid I don?t have much knowledge of the comm interface - is there a document I could be pointed out that lays out the protocol? Does it sound possible to send a pure array to JS in the way I?m doing in the landmarkerio-server? Best, James ? Sent from Mailbox On Wed, Aug 27, 2014 at 9:53 PM, Cyrille Rossant wrote: > We have pretty similar requirements for Vispy, as we target fast > visualization of big datasets using OpenGL in the IPython notebook. In > particular, we can't use Canvas or SVG because it's just too slow for > big data, so we have to use WebGL. This lets us leverage the GPU for > visualization. > I would be very interested in seeing the answers to your questions. > Notably, is it possible to use binary sockets for transferring large > amounts of binary data between Python and JavaScript? > Cyrille > 2014-08-27 20:52 GMT+01:00 Matthew Brett : >> Guys / gals, >> >> I want to ask for advice about writing a brain image display widget for >> IPython. >> >> I would like to make an IPython widget that can take an in-memory numpy array >> and do an interactive display of orthogonal slices from the array. The >> display will look something like Papaya: >> >> http://rii.uthscsa.edu/mango/papaya >> >> where clicking or moving the mouse causes matching slices to be displayed >> through the three axes of the numpy array (here a brain image). >> >> Papaya is pure javascript, so I am assuming that it loads the whole array >> (brain image) into a javascript variable and takes slices from that. >> >> What I would like to do, is to be able to keep the whole 3D array only in >> Python, and pass the slices as needed to a javascript viewer. >> >> In my ignorance, I am not sure which approach to go for first. >> >> Should I use the comm / widget interface for this? In that case I guess the >> procedure would be: >> >> * mouse movement generates a 'need slice' message from javascript >> * python kernel accepts 'need slice' message, takes slice from array, base64 >> encodes into JSON, sends 'here is your slice' message back to javascript >> with the data >> * javascript accepts message, unpacks base64'ed JSONed slice into variable and >> displays slice variable >> >> Is that right? I guess that involves Is there any chance that this >> will be fast enough for >> satisfying interactive movement through the image, which would likely require >> something like 20 slices per second, each of say 64 x 64 floating point? >> >> If not - is there something else I should look at instead? >> >> Another question for more thanks - should I use a Canvas or SVG element to >> display the images? The fabric.js README [1] seems to imply the >> Canvas element is >> faster for some interactive stuff, does anyone have relevant experience to >> share? >> >> Thanks a lot, >> >> Matthew >> >> [1] https://github.com/kangax/fabric.js/#history >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From Sebastien.Hinderer at inria.fr Thu Aug 28 04:05:15 2014 From: Sebastien.Hinderer at inria.fr (=?utf-8?Q?S=C3=A9bastien?= Hinderer) Date: Thu, 28 Aug 2014 10:05:15 +0200 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: References: Message-ID: <20140828080515.GB2951@pl-59055.rocqadm.inria.fr> Dear all, First of all, let me thank all those who have provided so useful responses to the question about setting up a multi-user notebook server. Basically, we will do it as all of you seem to have done it: creating a Unix account for each user so that security relies on Unix permissions. Technically, we will write our own scripts but they will definitely devrive from existing work, so thanks again for having provided valuable resources. One issue I'm curious about and that did not seem to be addressed in the resources I explored is the termination of webserver and kernel processes. Since we develop the solution in the context of a MOOC, there is the possibility of having plenty of users. So, letting all the processes run forever does not seem an option. There is indeed a limit in how many processes can run simultaneously on a system and, perhaps even more importantly, the number of available TCP ports is even smaller than that. Of course, the transpor beween web servers and kernels can be set-up to use Unix local sockets, but the webservs will all need a different port so if they are kept alive that means a limit on the number of users. Perhaps one possibility here would be to have a web server running in front and redirecting connections to the notebook severs, based on a URL prefix or a similar solution. Not sure whether/ how this can be implemented, though. Another suggestion, regarding IPython 3.0, would be to provide configuration parameters to let a kernel and a web server stop after some given idle time, or something like that. Any comment on this aspect will of course be welcome! Bes wishes, S?bastien. From bussonniermatthias at gmail.com Thu Aug 28 04:46:36 2014 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Thu, 28 Aug 2014 09:46:36 +0100 Subject: [IPython-dev] javascript / Python communication for image viewer In-Reply-To: <1409211914670.efeba11e@Nodemailer> References: <1409211914670.efeba11e@Nodemailer> Message-ID: Hi all, You probably don?t want to use widget framework directly but more comms object. Binary data exchange will be possible with https://github.com/ipython/ipython/pull/6110 ? M Le 28 ao?t 2014 ? 08:45, James Booth a ?crit : > Hey Matthew, Cyrille, > > I?m developing a three.js-based web app for image/object annotation called landmarker.io (I work in computer vision/machine learning, getting high quality annotations of 3D and 2D data is often critical to training models). It?s located here > > www.landmarker.io (tool, launches in demo by default) > https://github.com/menpo/landmarker.io (code) > > I?m also one of the developer?s of Menpo (www.menpo.io), which is a Python package for building and testing deformable models. I?m going to be developing an IPython widget-version of the landmarker.io tool, so we can correct annotations as we sport problems in the notebook. > > I mention all this as: > > 1. I?m also interested in understanding how we can transfer large arrays efficiently to IPython widgets > 2. I have a little experience in how best to handle this in a traditional client/server model, but I don?t know how well this will translate to the comm interface. > > Basically, landmarker.io expects to be able to talk to a RESTful interface that is implemented currently by: > > https://github.com/menpo/landmarkerio-server > > Originally, I sent meshes as large JSON objects, but this was pretty slow as: > > 1. It?s more work in JS to parse the JSON into an array > 2. It?s more work to turn this into the most efficient JS type for numerical arrays (which WebGL loves) which is an ArrayBuffer. > > https://developer.mozilla.org/en-US/docs/Web/API/ArrayBuffer > https://developer.mozilla.org/en-US/docs/Web/API/Float32Array > > This meant that even with a server on localhost, skipping through faces to annotate felt a little sluggish. > > Instead now I just directly build a pure ArrayBuffer in Python on the server and ship it to the client. That happens here in the server: > > https://github.com/menpo/landmarkerio-server/blob/master/landmarkerio/cache.py#L165 > > (I?m a little lazy here and literally save the file to disk, then gzip it. It could be done in memory with StringIO through, but I do this as a caching process so it?s going to disk anyway). > > On the client side, I make an XMLHttpRequest and demand an array buffer from the server: > https://github.com/menpo/landmarker.io/blob/master/src/js/app/lib/get.js#L9 > > For completion, the actual parsing of the array buffer is done here: > https://github.com/menpo/landmarker.io/blob/master/src/js/app/model/mesh.js#L104 > > Because three.js supports ArrayBuffers, this is really fast. I just point three to the array and we are away. With this implementation, browsing through subjects to annotate with a server on localhost (akin to moving between slices in your brain scan I imagine) is very fast. > > I?m afraid I don?t have much knowledge of the comm interface - is there a document I could be pointed out that lays out the protocol? Does it sound possible to send a pure array to JS in the way I?m doing in the landmarkerio-server? > > Best, > James > > > ? > Sent from Mailbox > > > On Wed, Aug 27, 2014 at 9:53 PM, Cyrille Rossant wrote: > > We have pretty similar requirements for Vispy, as we target fast > visualization of big datasets using OpenGL in the IPython notebook. In > particular, we can't use Canvas or SVG because it's just too slow for > big data, so we have to use WebGL. This lets us leverage the GPU for > visualization. > > I would be very interested in seeing the answers to your questions. > Notably, is it possible to use binary sockets for transferring large > amounts of binary data between Python and JavaScript? > > Cyrille > > 2014-08-27 20:52 GMT+01:00 Matthew Brett : > > Guys / gals, > > > > I want to ask for advice about writing a brain image display widget for > > IPython. > > > > I would like to make an IPython widget that can take an in-memory numpy array > > and do an interactive display of orthogonal slices from the array. The > > display will look something like Papaya: > > > > http://rii.uthscsa.edu/mango/papaya > > > > where clicking or moving the mouse causes matching slices to be displayed > > through the three axes of the numpy array (here a brain image). > > > > Papaya is pure javascript, so I am assuming that it loads the whole array > > (brain image) into a javascript variable and takes slices from that. > > > > What I would like to do, is to be able to keep the whole 3D array only in > > Python, and pass the slices as needed to a javascript viewer. > > > > In my ignorance, I am not sure which approach to go for first. > > > > Should I use the comm / widget interface for this? In that case I guess the > > procedure would be: > > > > * mouse movement generates a 'need slice' message from javascript > > * python kernel accepts 'need slice' message, takes slice from array, base64 > > encodes into JSON, sends 'here is your slice' message back to javascript > > with the data > > * javascript accepts message, unpacks base64'ed JSONed slice into variable and > > displays slice variable > > > > Is that right? I guess that involves Is there any chance that this > > will be fast enough for > > satisfying interactive movement through the image, which would likely require > > something like 20 slices per second, each of say 64 x 64 floating point? > > > > If not - is there something else I should look at instead? > > > > Another question for more thanks - should I use a Canvas or SVG element to > > display the images? The fabric.js README [1] seems to imply the > > Canvas element is > > faster for some interactive stuff, does anyone have relevant experience to > > share? > > > > Thanks a lot, > > > > Matthew > > > > [1] https://github.com/kangax/fabric.js/#history > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Thu Aug 28 05:07:37 2014 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Thu, 28 Aug 2014 10:07:37 +0100 Subject: [IPython-dev] Using IPython notebooks to teach Python In-Reply-To: <20140828080515.GB2951@pl-59055.rocqadm.inria.fr> References: <20140828080515.GB2951@pl-59055.rocqadm.inria.fr> Message-ID: <89D82AF9-4DCF-42F5-85DC-869FD98D5F72@gmail.com> Hi, Please have a look at https://github.com/jupyter/jupyterhub and https://github.com/jupyter/configurable-http-proxy This is what will be used for IPython multi-user. Le 28 ao?t 2014 ? 09:05, S?bastien Hinderer a ?crit : > Dear all, > > First of all, let me thank all those who have provided so useful > responses to the question about setting up a multi-user notebook server. > > Basically, we will do it as all of you seem to have done it: creating a > Unix account for each user so that security relies on Unix permissions. > Technically, we will write our own scripts but they will definitely > devrive from existing work, so thanks again for having provided > valuable resources. > > One issue I'm curious about and that did not seem to be addressed in the > resources I explored is the termination of webserver and > kernel processes. Since we develop the solution in the context of a > MOOC, there is the possibility of having plenty of users. So, letting > all the processes run forever does not seem an option. There is > indeed a limit in how many processes can run simultaneously on a system > and, perhaps even more importantly, the number of available TCP ports is > even smaller than that. Of course, the transpor beween web servers and > kernels can be set-up to use Unix local sockets, but the webservs will > all need a different port so if they are kept alive that means a limit > on the number of users. > > Perhaps one possibility here would be to have a web server running in > front and redirecting connections to the notebook severs, based on a URL > prefix or a similar solution. Not sure whether/ how this can be > implemented, though. This is what above link do. > Another suggestion, regarding IPython 3.0, would be to provide > configuration parameters to let a kernel and a web server stop after > some given idle time, or something like that. Probably not. as you are in control of the server/proxy, you can have a deamon that monitor the connexion and send the rest command to shutdown with the logic you like. It will be much more reliable and flexible than baking in all the imaginable logic in IPython. > Any comment on this aspect will of course be welcome! We will be happy to get help on the developpement of this, The current funding of IPython is (as far as I know) coming to an end at the end of this year, so we cannot promise that things will continue to move fast in the future. If your MOOC as a lot of audience and you use IPython/Python a lot, please consider proposing your hierarchy to give something to Python community. Be aware that isolating user on the server-side using container or alike is not the hard part of the security issue in using IPython. The hard part is on the browser side. To anyone that say that container are used on server-side for security, be aware that what you say is equivalent to say : ? The server of your bank is in a vault with 5 ton door, of course you can send plain text email with your credit card number to any people that claim to be the server, no one can access this server ? The risk is js injection in the browser, if it is a MOOC, steeling credential, replaying cookie auth, CORS on web socket, and so on and so forth. ? M > > Bes wishes, > S?bastien. > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.hirschfeld at gmail.com Thu Aug 28 06:42:41 2014 From: dave.hirschfeld at gmail.com (Dave Hirschfeld) Date: Thu, 28 Aug 2014 10:42:41 +0000 (UTC) Subject: [IPython-dev] =?utf-8?q?AttributeError=3A_=27Kernel=27_object_has?= =?utf-8?q?_no_attribute_=27abort=5Fqueues=27?= Message-ID: I got the aforementioned error in an ipengine process. I'm not sure of the sequence of events which led up to it, but from what I can see it doesn't exist in 2.1. [IPEngineApp] ERROR | Exception in control handler: Traceback (most recent call last): File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site- packages\IPython\kernel\zmq\ipkernel.py", line 200, in dispatch_control handler(self.control_stream, idents, msg) File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site- packages\IPython\kernel\zmq\ipkernel.py", line 666, in abort_request self.abort_queues() AttributeError: 'Kernel' object has no attribute 'abort_queues' https://github.com/ipython/ipython/blob/rel- 2.1.0/IPython/kernel/zmq/ipkernel.py I've confirmed that this bug is also present in master: https://github.com/ipython/ipython/blob/4dcc51a102619862141ee68742117adb4cd1e5 1d/IPython/kernel/zmq/kernelbase.py PR available @ https://github.com/ipython/ipython/pull/6374 HTH, Dave From benjaminrk at gmail.com Thu Aug 28 14:03:12 2014 From: benjaminrk at gmail.com (MinRK) Date: Thu, 28 Aug 2014 11:03:12 -0700 Subject: [IPython-dev] AttributeError: 'Kernel' object has no attribute 'abort_queues' In-Reply-To: References: Message-ID: Thanks for catching it On Thu, Aug 28, 2014 at 3:42 AM, Dave Hirschfeld wrote: > I got the aforementioned error in an ipengine process. > I'm not sure of the sequence of events which led up to it, but from what I > can > see it doesn't exist in 2.1. > > [IPEngineApp] ERROR | Exception in control handler: > Traceback (most recent call last): > File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site- > packages\IPython\kernel\zmq\ipkernel.py", line 200, in dispatch_control > handler(self.control_stream, idents, msg) > File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site- > packages\IPython\kernel\zmq\ipkernel.py", line 666, in abort_request > self.abort_queues() > AttributeError: 'Kernel' object has no attribute 'abort_queues' > > https://github.com/ipython/ipython/blob/rel- > 2.1.0/IPython/kernel/zmq/ipkernel.py > > I've confirmed that this bug is also present in master: > > > https://github.com/ipython/ipython/blob/4dcc51a102619862141ee68742117adb4cd1e5 > 1d/IPython/kernel/zmq/kernelbase.py > > > PR available @ https://github.com/ipython/ipython/pull/6374 > > > HTH, > Dave > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Thu Aug 28 14:03:12 2014 From: benjaminrk at gmail.com (MinRK) Date: Thu, 28 Aug 2014 11:03:12 -0700 Subject: [IPython-dev] AttributeError: 'Kernel' object has no attribute 'abort_queues' In-Reply-To: References: Message-ID: Thanks for catching it On Thu, Aug 28, 2014 at 3:42 AM, Dave Hirschfeld wrote: > I got the aforementioned error in an ipengine process. > I'm not sure of the sequence of events which led up to it, but from what I > can > see it doesn't exist in 2.1. > > [IPEngineApp] ERROR | Exception in control handler: > Traceback (most recent call last): > File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site- > packages\IPython\kernel\zmq\ipkernel.py", line 200, in dispatch_control > handler(self.control_stream, idents, msg) > File "C:\Python\envs\quantdev\1.4.0.post673.g1d57b50\lib\site- > packages\IPython\kernel\zmq\ipkernel.py", line 666, in abort_request > self.abort_queues() > AttributeError: 'Kernel' object has no attribute 'abort_queues' > > https://github.com/ipython/ipython/blob/rel- > 2.1.0/IPython/kernel/zmq/ipkernel.py > > I've confirmed that this bug is also present in master: > > > https://github.com/ipython/ipython/blob/4dcc51a102619862141ee68742117adb4cd1e5 > 1d/IPython/kernel/zmq/kernelbase.py > > > PR available @ https://github.com/ipython/ipython/pull/6374 > > > HTH, > Dave > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Thu Aug 28 18:44:22 2014 From: benjaminrk at gmail.com (MinRK) Date: Thu, 28 Aug 2014 15:44:22 -0700 Subject: [IPython-dev] nbformat updates Message-ID: Hello all, IPython 3.0 will carry major updates to both the kernel message spec and the notebook document format . The message spec changes landed months ago, and include shims to continue supporting kernels that talk msg spec v4 via adapters. We will backport the new nbformat to 2.3 once it is finalized, so that 2.x users will still be able to read IPython notebooks authored in 3.0 (they will be downgraded to the old nbformat on open). The question under discussion right now: Do we need shims like those in the message specification in the notebook format? The main place where the nbformat is a public API is in nbconvert, where custom templates or preprocessors need to know the nbformat structure. Code such as Jake Vanderplas? liquid_tags for pelican will need to be updated to work with IPython 3.0. Shims for nbformat would be a lot more complicated than for the message spec, so I want to gauge interest / need before investing too much time in it. -MinRK ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jklymak at gmail.com Fri Aug 29 12:00:50 2014 From: jklymak at gmail.com (Klymak Jody) Date: Fri, 29 Aug 2014 09:00:50 -0700 Subject: [IPython-dev] Latex highlighting for markdown cells? Message-ID: Sorry if I've double posted this - I keep sending from the wrong account... Hi all, I am using a lot of tags in my markdown cells, and that works great for what I am writing. However, is there a way to get latex-style highlighting in my cells rather than markdown (or some combination of both)? As soon as you put an underscore in the cell, everything until the next underscore is rendered as oblique, which is not very easy to read. i.e. if I type: Hello $a_b$ there. Everything after the "b" is oblique. Thanks for any pointers. Cheers, Jody From nathan12343 at gmail.com Fri Aug 29 12:04:48 2014 From: nathan12343 at gmail.com (Nathan Goldbaum) Date: Fri, 29 Aug 2014 09:04:48 -0700 Subject: [IPython-dev] Latex highlighting for markdown cells? In-Reply-To: References: Message-ID: This is a known issue I raised in January: https://github.com/ipython/ipython/issues/4850 I get the impression from Mon's and Brian's comments that this would be not very straightforward to fix. On Friday, August 29, 2014, Klymak Jody wrote: > Sorry if I've double posted this - I keep sending from the wrong account... > > Hi all, > > I am using a lot of tags in my markdown cells, and that works > great for what I am writing. However, is there a way to get latex-style > highlighting in my cells rather than markdown (or some combination of > both)? As soon as you put an underscore in the cell, everything until the > next underscore is rendered as oblique, which is not very easy to read. > > i.e. if I type: > > Hello $a_b$ there. > > Everything after the "b" is oblique. > > Thanks for any pointers. > > Cheers, Jody > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Fri Aug 29 12:05:42 2014 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Fri, 29 Aug 2014 17:05:42 +0100 Subject: [IPython-dev] Latex highlighting for markdown cells? In-Reply-To: References: Message-ID: <4C684A7E-7FEC-4D25-8E5A-BA05EDFD46C5@gmail.com> Le 29 ao?t 2014 ? 17:00, Klymak Jody a ?crit : > Sorry if I've double posted this - I keep sending from the wrong account... > > Hi all, > > I am using a lot of tags in my markdown cells, and that works great for what I am writing. However, is there a way to get latex-style highlighting in my cells rather than markdown (or some combination of both)? As soon as you put an underscore in the cell, everything until the next underscore is rendered as oblique, which is not very easy to read. > Someone would have to write a markdown+latex mode in code mirror that detect $/$$ et switch to latex. It would have the advantage of also highlighting latex correctly. But it need to be done. is not hard, but not trivial either. ? M > i.e. if I type: > > Hello $a_b$ there. > > Everything after the "b" is oblique. > > Thanks for any pointers. > > Cheers, Jody > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From zvoros at gmail.com Fri Aug 29 13:38:25 2014 From: zvoros at gmail.com (=?windows-1252?Q?Zolt=E1n_V=F6r=F6s?=) Date: Fri, 29 Aug 2014 19:38:25 +0200 Subject: [IPython-dev] Latex highlighting for markdown cells? In-Reply-To: <4C684A7E-7FEC-4D25-8E5A-BA05EDFD46C5@gmail.com> References: <4C684A7E-7FEC-4D25-8E5A-BA05EDFD46C5@gmail.com> Message-ID: <5400BA91.1090202@gmail.com> Hi Matthias, I have recently been wondering myself about this. What I wanted to do is insert the closing \end{environment} tags in the markdown cell, and insert e.g., \frac{}{} and move the cursor back to the first curly bracket, if the user types \fra, but syntax highlighting would also be useful. Could you point me to some sort of documentation as to how one starts out writing a plugin for this? I would be willing to work on this. Cheers, Zolt?n On 08/29/2014 06:05 PM, Matthias Bussonnier wrote: > Le 29 ao?t 2014 ? 17:00, Klymak Jody a ?crit : > >> Sorry if I've double posted this - I keep sending from the wrong account... >> >> Hi all, >> >> I am using a lot of tags in my markdown cells, and that works great for what I am writing. However, is there a way to get latex-style highlighting in my cells rather than markdown (or some combination of both)? As soon as you put an underscore in the cell, everything until the next underscore is rendered as oblique, which is not very easy to read. >> > Someone would have to write a markdown+latex mode in code mirror that detect $/$$ et switch to latex. > It would have the advantage of also highlighting latex correctly. > But it need to be done. > > is not hard, but not trivial either. > ? > M > From cyrille.rossant at gmail.com Sat Aug 30 08:57:45 2014 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Sat, 30 Aug 2014 13:57:45 +0100 Subject: [IPython-dev] ANN: Vispy 0.3, high-performance visualization in Python Message-ID: Hi, We are pleased to announce the release of Vispy version 0.3! This release provides new graphics API (including a scene graph), a basic OpenGL backend for matplotlib, and experimental integration in the IPython notebook. Further, we have improved the existing modules and added several application backends (e.g. wx and sdl2). We are now focusing our efforts on: * Consolidating the higher-level graphics APIs: visuals, transforms, scene graph. * Improving the performance of the scene graph, notably through batch rendering (Collections system). * Adding more features to the matplotlib backend. * Improving the integration in the IPython notebook: better performance, improved event loop integration, client-side backend with WebGL. Eventually, scientists with no knowledge of OpenGL will be able to easily create fast and scalable interactive visualizations. We will greatly appreciate any feedback, bug reports, and feature requests. See http://vispy.org for downloads and more information. The Vispy team From tom at hackerschool.com Sat Aug 30 12:32:16 2014 From: tom at hackerschool.com (Thomas Ballinger) Date: Sat, 30 Aug 2014 09:32:16 -0700 Subject: [IPython-dev] __future__ statements in ipython -i test.py Message-ID: While addressing a bug in bpython, I found that (terminal) ipython and python differ in behavior when a file test.py contains from __future__ import division and I run (i)python -i test.py and then enter 1/2. The behavior is the same for $ (i)python -i -c 'from __future__ import division' and then doing some division. Before I look too far into if fixing the problem in bpython is possible, I wanted to see if I could crib off of your collective experience with this. Is this behavior difficult to implement, not actually desirable, or just not something someone has gotten around to? After Thomas's excellent answers last time to my encoding questions I tend to think you all have dealt with any problem I face a few times already and thought about it more than I have. :) Tom -------------- next part -------------- An HTML attachment was scrubbed... URL: From hughesadam87 at gmail.com Sat Aug 30 12:58:40 2014 From: hughesadam87 at gmail.com (Adam Hughes) Date: Sat, 30 Aug 2014 12:58:40 -0400 Subject: [IPython-dev] ANN: Vispy 0.3, high-performance visualization in Python In-Reply-To: References: Message-ID: Looks really awesome, thanks for sharing! On Sat, Aug 30, 2014 at 8:57 AM, Cyrille Rossant wrote: > Hi, > > We are pleased to announce the release of Vispy version 0.3! > > This release provides new graphics API (including a scene graph), a > basic OpenGL backend for matplotlib, and experimental integration in > the IPython notebook. Further, we have improved the existing modules > and added several application backends (e.g. wx and sdl2). > > We are now focusing our efforts on: > > * Consolidating the higher-level graphics APIs: visuals, transforms, > scene graph. > * Improving the performance of the scene graph, notably through batch > rendering (Collections system). > * Adding more features to the matplotlib backend. > * Improving the integration in the IPython notebook: better > performance, improved event loop integration, client-side backend with > WebGL. > > Eventually, scientists with no knowledge of OpenGL will be able to > easily create fast and scalable interactive visualizations. > > We will greatly appreciate any feedback, bug reports, and feature > requests. See http://vispy.org for downloads and more information. > > The Vispy team > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hughesadam87 at gmail.com Sat Aug 30 13:41:30 2014 From: hughesadam87 at gmail.com (Adam Hughes) Date: Sat, 30 Aug 2014 13:41:30 -0400 Subject: [IPython-dev] Widget example that uses classes instead of globals? Message-ID: Hi, This question follows directly from this recent thread on the feasiblity of widgets ( http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-td5068197.html) but I thought was specific enough to be a new thread. In all of the notebook examples I've seen, a GUI is defined in its own cell with global variables that are shared between functions. I'd really like to see a complete, simple example of a GUI build as a class, where most of the events, updates and default values are set within the class. And then when the class is instantiated, the notebook somehow can still output the GUI. For example, would something of this design be compatible with the current widget design pattern: *class Foo(object):* * variable = defaultvalue* * def update_variable(self):* * ....* * def redraw(self):* * ...* *# Instantiate the class* *f = Foo()* *f.variable_widget.on_trait_change(update_variable, 'value')* Or is this type of design pattern out of the question? We're trying to move in this direction so that we can compartmentalize and template as much behavior as possible, and make a simple framework for building up more complex GUIs. It's just now clear to me if such a pattern is possible. If anyone has any insights or examples, would you mind sharing? THanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From hughesadam87 at gmail.com Sat Aug 30 13:43:26 2014 From: hughesadam87 at gmail.com (Adam Hughes) Date: Sat, 30 Aug 2014 13:43:26 -0400 Subject: [IPython-dev] Widget example that uses classes instead of globals? In-Reply-To: References: Message-ID: And just to clarify, I presume that the notebook is actively "looking" for particular functions or something in the cell that let's the notebook know "hey, this is widget/GUI code, I need to output it as so". I had worried that if this critical piece was buried in a class, the notebook wouldn't recognize it, and the GUI wouldn't display. On Sat, Aug 30, 2014 at 1:41 PM, Adam Hughes wrote: > Hi, > > This question follows directly from this recent thread on the feasiblity > of widgets ( > http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-td5068197.html) > but I thought was specific enough to be a new thread. > > In all of the notebook examples I've seen, a GUI is defined in its own > cell with global variables that are shared between functions. I'd really > like to see a complete, simple example of a GUI build as a class, where > most of the events, updates and default values are set within the class. > And then when the class is instantiated, the notebook somehow can still > output the GUI. > > For example, would something of this design be compatible with the current > widget design pattern: > > *class Foo(object):* > > * variable = defaultvalue* > > * def update_variable(self):* > * ....* > > * def redraw(self):* > * ...* > > *# Instantiate the class* > *f = Foo()* > *f.variable_widget.on_trait_change(update_variable, 'value')* > > Or is this type of design pattern out of the question? We're trying to > move in this direction so that we can compartmentalize and template as much > behavior as possible, and make a simple framework for building up more > complex GUIs. It's just now clear to me if such a pattern is possible. If > anyone has any insights or examples, would you mind sharing? > > THanks > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cyrille.rossant at gmail.com Sat Aug 30 13:49:49 2014 From: cyrille.rossant at gmail.com (Cyrille Rossant) Date: Sat, 30 Aug 2014 10:49:49 -0700 Subject: [IPython-dev] Widget example that uses classes instead of globals? Message-ID: <-1189340719239998470@unknownmsgid> I think you can use the "display" function to render a widget instance, does that answer your question? ------------------------------ De : Adam Hughes Envoy? : ?30/?08/?2014 18:43 ? : IPython developers list Objet : Re: [IPython-dev] Widget example that uses classes instead of globals? And just to clarify, I presume that the notebook is actively "looking" for particular functions or something in the cell that let's the notebook know "hey, this is widget/GUI code, I need to output it as so". I had worried that if this critical piece was buried in a class, the notebook wouldn't recognize it, and the GUI wouldn't display. On Sat, Aug 30, 2014 at 1:41 PM, Adam Hughes wrote: > Hi, > > This question follows directly from this recent thread on the feasiblity > of widgets ( > http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-td5068197.html) > but I thought was specific enough to be a new thread. > > In all of the notebook examples I've seen, a GUI is defined in its own > cell with global variables that are shared between functions. I'd really > like to see a complete, simple example of a GUI build as a class, where > most of the events, updates and default values are set within the class. > And then when the class is instantiated, the notebook somehow can still > output the GUI. > > For example, would something of this design be compatible with the current > widget design pattern: > > *class Foo(object):* > > * variable = defaultvalue* > > * def update_variable(self):* > * ....* > > * def redraw(self):* > * ...* > > *# Instantiate the class* > *f = Foo()* > *f.variable_widget.on_trait_change(update_variable, 'value')* > > Or is this type of design pattern out of the question? We're trying to > move in this direction so that we can compartmentalize and template as much > behavior as possible, and make a simple framework for building up more > complex GUIs. It's just now clear to me if such a pattern is possible. If > anyone has any insights or examples, would you mind sharing? > > THanks > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tritemio at gmail.com Sat Aug 30 14:21:33 2014 From: tritemio at gmail.com (Antonino Ingargiola) Date: Sat, 30 Aug 2014 11:21:33 -0700 Subject: [IPython-dev] Widget example that uses classes instead of globals? In-Reply-To: <-1189340719239998470@unknownmsgid> References: <-1189340719239998470@unknownmsgid> Message-ID: Hi, Sorry for entering in the discussion on-fly... My question is more specific (but possibly very similar). Would be possible to call interact with a class object (i.e. a callable class) instead of a function? This would allow to store all the GUI specific state in the class (for example precomputing figures) without the need of using globals. When I quickly tried (with ipython 2.2), interact returned an error complaining about the number of parameters passed to the object. So I resorted to using globals. Below I attached the code I currently use to pre-compute a set of figures (a slider selects which figure to display). I would appreciate any comment on possibly improving this pattern. fig_cache = {} def plot_mfit_i(ich=0, nodisplay=False): global fig_cache if ich not in fig_cache: fig = mfit.plot_mfit(fitter, ich=ich, residuals=True, return_fig=True) fig_cache[ich] = fig plt.close(fig) if not nodisplay: display(fig_cache[ich]) for i in range(8): plot_mfit_i(i, nodisplay=True) # Precompute all the figures interact(plot_mfit_i, ich=(0, 7), nodisplay=fixed(False)); # display the figure selected through the slider -- Antonio On Sat, Aug 30, 2014 at 10:49 AM, Cyrille Rossant wrote: > I think you can use the "display" function to render a widget instance, > does that answer your question? > ------------------------------ > De : Adam Hughes > Envoy? : ?30/?08/?2014 18:43 > ? : IPython developers list > Objet : Re: [IPython-dev] Widget example that uses classes instead of > globals? > > And just to clarify, I presume that the notebook is actively "looking" for > particular functions or something in the cell that let's the notebook know > "hey, this is widget/GUI code, I need to output it as so". I had worried > that if this critical piece was buried in a class, the notebook wouldn't > recognize it, and the GUI wouldn't display. > > > On Sat, Aug 30, 2014 at 1:41 PM, Adam Hughes > wrote: > >> Hi, >> >> This question follows directly from this recent thread on the feasiblity >> of widgets ( >> http://python.6.x6.nabble.com/Feasilibity-of-widgets-layout-for-mimicking-this-td5068197.html) >> but I thought was specific enough to be a new thread. >> >> In all of the notebook examples I've seen, a GUI is defined in its own >> cell with global variables that are shared between functions. I'd really >> like to see a complete, simple example of a GUI build as a class, where >> most of the events, updates and default values are set within the class. >> And then when the class is instantiated, the notebook somehow can still >> output the GUI. >> >> For example, would something of this design be compatible with the >> current widget design pattern: >> >> *class Foo(object):* >> >> * variable = defaultvalue* >> >> * def update_variable(self):* >> * ....* >> >> * def redraw(self):* >> * ...* >> >> *# Instantiate the class* >> *f = Foo()* >> *f.variable_widget.on_trait_change(update_variable, 'value')* >> >> Or is this type of design pattern out of the question? We're trying to >> move in this direction so that we can compartmentalize and template as much >> behavior as possible, and make a simple framework for building up more >> complex GUIs. It's just now clear to me if such a pattern is possible. If >> anyone has any insights or examples, would you mind sharing? >> >> THanks >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirage007 at gmail.com Sat Aug 30 21:36:28 2014 From: mirage007 at gmail.com (Ivan Zhang) Date: Sat, 30 Aug 2014 21:36:28 -0400 Subject: [IPython-dev] Runtime Console Using IPython Message-ID: Hi All, As a way to familiarize myself with IPython, I wrote a simple tool that allows an IPython Kernel to be run as part of a thread, effectively creating a way to remotely monitor and change the state of a running application in real time. There are a few items that I have yet to figure out, I am hoping for some feedback from everyone on whether there is a better approach, or an easy way to fix some of the limitations. I wrote a post describing the specifics of the tool here: http://mirage007.github.io/runtime/console/python/ipython/2014/08/28/rtconsole/ and the source code is located here: https://github.com/mirage007/rtconsole Any feedback would be greatly appreciated! Thanks, -- Ivan -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Aug 30 22:50:30 2014 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 30 Aug 2014 19:50:30 -0700 Subject: [IPython-dev] Runtime Console Using IPython In-Reply-To: References: Message-ID: Hi Ivan, On Sat, Aug 30, 2014 at 6:36 PM, Ivan Zhang wrote: > Hi All, > > As a way to familiarize myself with IPython, I wrote a simple tool that > allows an IPython Kernel to be run as part of a thread, effectively > creating a way to remotely monitor and change the state of a running > application in real time. > Great! Thanks for sharing that... I'm sorry that I haven't looked at your code in detail, but along those lines, I thought at least it would be useful to point you to these examples: https://github.com/ipython/ipython/tree/master/examples/Embedding They illustrate how to embed a kernel in a process that may have its own event loop (Qt and Wx provided), and how to connect to it from an external Qt console, with both the Qt console and the GUI controls manipulating the same namespace. It's a different take on the same kind of ideas, and you may find it useful. Cheers f -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason-sage at creativetrax.com Sat Aug 30 23:25:46 2014 From: jason-sage at creativetrax.com (Jason Grout) Date: Sat, 30 Aug 2014 23:25:46 -0400 Subject: [IPython-dev] Runtime Console Using IPython In-Reply-To: References: Message-ID: <540295BA.6000802@creativetrax.com> On 8/30/14, 22:50, Fernando Perez wrote: > They illustrate how to embed a kernel in a process that may have its own > event loop (Qt and Wx provided), and how to connect to it from an > external Qt console, with both the Qt console and the GUI controls > manipulating the same namespace. > > It's a different take on the same kind of ideas, and you may find it useful. > Related to this, I was thinking the other day that it would be really useful if there was an easy standard way to make small, short widget control update messages and resulting event triggers do their work on a separate thread. I had some usecases the other day where the interactivity was severely lagging because the kernel was tied up doing other unrelated work. This probably wouldn't make sense to do by default, but maybe with some special decorator or something. I experimented with this sort of thing several years ago with Sage interacts, but I'm thinking through it in the context of the IPython widgets now. Just thought I'd bring it up as a related idea to your paragraph above. Thanks, Jason From wes.turner at gmail.com Sun Aug 31 08:46:16 2014 From: wes.turner at gmail.com (Wes Turner) Date: Sun, 31 Aug 2014 07:46:16 -0500 Subject: [IPython-dev] ANN: Vispy 0.3, high-performance visualization in Python In-Reply-To: References: Message-ID: Looks cool from afar! A few posts to reddit may also find your audience: * http://www.reddit.com/r/python * http://www.reddit.com/r/visualization * http://www.reddit.com/r/IPython On Aug 30, 2014 7:57 AM, "Cyrille Rossant" wrote: > Hi, > > We are pleased to announce the release of Vispy version 0.3! > > This release provides new graphics API (including a scene graph), a > basic OpenGL backend for matplotlib, and experimental integration in > the IPython notebook. Further, we have improved the existing modules > and added several application backends (e.g. wx and sdl2). > > We are now focusing our efforts on: > > * Consolidating the higher-level graphics APIs: visuals, transforms, > scene graph. > * Improving the performance of the scene graph, notably through batch > rendering (Collections system). > * Adding more features to the matplotlib backend. > * Improving the integration in the IPython notebook: better > performance, improved event loop integration, client-side backend with > WebGL. > > Eventually, scientists with no knowledge of OpenGL will be able to > easily create fast and scalable interactive visualizations. > > We will greatly appreciate any feedback, bug reports, and feature > requests. See http://vispy.org for downloads and more information. > > The Vispy team > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmeurer at gmail.com Sun Aug 31 12:51:15 2014 From: asmeurer at gmail.com (Aaron Meurer) Date: Sun, 31 Aug 2014 11:51:15 -0500 Subject: [IPython-dev] __future__ statements in ipython -i test.py In-Reply-To: References: Message-ID: <-7772341799178964587@unknownmsgid> I believe this has been discussed before on this list at least in the context of %run. Aaron Meurer > On Aug 30, 2014, at 11:32 AM, Thomas Ballinger wrote: > > While addressing a bug in bpython, I found that (terminal) ipython and python differ in behavior when a file test.py contains > > from __future__ import division > > and I run (i)python -i test.py and then enter 1/2. The behavior is the same for > > $ (i)python -i -c 'from __future__ import division' > > and then doing some division. > > Before I look too far into if fixing the problem in bpython is possible, I wanted to see if I could crib off of your collective experience with this. Is this behavior difficult to implement, not actually desirable, or just not something someone has gotten around to? After Thomas's excellent answers last time to my encoding questions I tend to think you all have dealt with any problem I face a few times already and thought about it more than I have. :) > > Tom > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From asmeurer at gmail.com Sun Aug 31 14:07:59 2014 From: asmeurer at gmail.com (Aaron Meurer) Date: Sun, 31 Aug 2014 13:07:59 -0500 Subject: [IPython-dev] __future__ statements in ipython -i test.py In-Reply-To: <-7772341799178964587@unknownmsgid> References: <-7772341799178964587@unknownmsgid> Message-ID: http://mail.scipy.org/pipermail/ipython-dev/2012-April/008956.html On Sun, Aug 31, 2014 at 11:51 AM, Aaron Meurer wrote: > I believe this has been discussed before on this list at least in the > context of %run. > > Aaron Meurer > > > On Aug 30, 2014, at 11:32 AM, Thomas Ballinger > wrote: > > > > While addressing a bug in bpython, I found that (terminal) ipython and > python differ in behavior when a file test.py contains > > > > from __future__ import division > > > > and I run (i)python -i test.py and then enter 1/2. The behavior is the > same for > > > > $ (i)python -i -c 'from __future__ import division' > > > > and then doing some division. > > > > Before I look too far into if fixing the problem in bpython is possible, > I wanted to see if I could crib off of your collective experience with > this. Is this behavior difficult to implement, not actually desirable, or > just not something someone has gotten around to? After Thomas's excellent > answers last time to my encoding questions I tend to think you all have > dealt with any problem I face a few times already and thought about it more > than I have. :) > > > > Tom > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Sun Aug 31 14:14:57 2014 From: bussonniermatthias at gmail.com (Matthias Bussonnier) Date: Sun, 31 Aug 2014 19:14:57 +0100 Subject: [IPython-dev] Latex highlighting for markdown cells? In-Reply-To: <5400BA91.1090202@gmail.com> References: <4C684A7E-7FEC-4D25-8E5A-BA05EDFD46C5@gmail.com> <5400BA91.1090202@gmail.com> Message-ID: <55FBA463-ED7D-499E-9293-0A5D90245861@gmail.com> Le 29 ao?t 2014 ? 18:38, Zolt?n V?r?s a ?crit : > Hi Matthias, > > I have recently been wondering myself about this. What I wanted to do is > insert the closing \end{environment} tags in the markdown cell, and > insert e.g., \frac{}{} and move the cursor back to the first curly > bracket, if the user types \fra, but syntax highlighting would also be > useful. Could you point me to some sort of documentation as to how one > starts out writing a plugin for this? I would be willing to work on this. Sorry for the short mail : http://codemirror.net/doc/manual.html#modeapi Have also a look at this part of code mirror doc : Sometimes, it is useful for modes to nest?to have one mode delegate work to another mode. An example of this kind of mode is the mixed-mode HTML mode. To implement such nesting, it is usually necessary to create mode objects and copy states yourself. To create a mode object, there are CodeMirror.getMode(options, parserConfig), where the first argument is a configuration object as passed to the mode constructor function, and the second argument is a mode specification as in the mode option. To copy a state object, call CodeMirror.copyState(mode, state), where mode is the mode that created the given state. ? > > Cheers, > Zolt?n > > On 08/29/2014 06:05 PM, Matthias Bussonnier wrote: >> Le 29 ao?t 2014 ? 17:00, Klymak Jody a ?crit : >> >>> Sorry if I've double posted this - I keep sending from the wrong account... >>> >>> Hi all, >>> >>> I am using a lot of tags in my markdown cells, and that works great for what I am writing. However, is there a way to get latex-style highlighting in my cells rather than markdown (or some combination of both)? As soon as you put an underscore in the cell, everything until the next underscore is rendered as oblique, which is not very easy to read. >>> >> Someone would have to write a markdown+latex mode in code mirror that detect $/$$ et switch to latex. >> It would have the advantage of also highlighting latex correctly. >> But it need to be done. >> >> is not hard, but not trivial either. >> ? >> M >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From doug.blank at gmail.com Sun Aug 31 14:37:49 2014 From: doug.blank at gmail.com (Doug Blank) Date: Sun, 31 Aug 2014 14:37:49 -0400 Subject: [IPython-dev] Latex highlighting for markdown cells? In-Reply-To: <55FBA463-ED7D-499E-9293-0A5D90245861@gmail.com> References: <4C684A7E-7FEC-4D25-8E5A-BA05EDFD46C5@gmail.com> <5400BA91.1090202@gmail.com> <55FBA463-ED7D-499E-9293-0A5D90245861@gmail.com> Message-ID: Here is an example of the codemirror overlay with IPython's mode: https://bitbucket.org/ipre/calico/src/master/notebooks/nbextensions/calico-spell-check.js That code is a little bit complicated as it has a function, makeOverlay, that makes a function overlay. That is so you can make spell-check overlays for as many modes as you want. BTW, this appears that it will need some changes to work with IPython3/Jupyter. Hope that helps, -Doug On Sun, Aug 31, 2014 at 2:14 PM, Matthias Bussonnier < bussonniermatthias at gmail.com> wrote: > > Le 29 ao?t 2014 ? 18:38, Zolt?n V?r?s a ?crit : > > Hi Matthias, > > I have recently been wondering myself about this. What I wanted to do is > insert the closing \end{environment} tags in the markdown cell, and > insert e.g., \frac{}{} and move the cursor back to the first curly > bracket, if the user types \fra, but syntax highlighting would also be > useful. Could you point me to some sort of documentation as to how one > starts out writing a plugin for this? I would be willing to work on this. > > > Sorry for the short mail : > > http://codemirror.net/doc/manual.html#modeapi > > Have also a look at this part of code mirror doc : > > Sometimes, it is useful for modes to nest?to have one mode delegate work > to another mode. An example of this kind of mode is the mixed-mode HTML > mode. To implement such nesting, it is usually necessary to create mode > objects and copy states yourself. To create a mode object, there > are CodeMirror.getMode(options, parserConfig), where the first argument is > a configuration object as passed to the mode constructor function, and the > second argument is a mode specification as in the mode option. To copy a > state object, call CodeMirror.copyState(mode, state), where mode is the > mode that created the given state. > > ? > > > > Cheers, > Zolt?n > > On 08/29/2014 06:05 PM, Matthias Bussonnier wrote: > > Le 29 ao?t 2014 ? 17:00, Klymak Jody a ?crit : > > Sorry if I've double posted this - I keep sending from the wrong account... > > Hi all, > > I am using a lot of tags in my markdown cells, and that works > great for what I am writing. However, is there a way to get latex-style > highlighting in my cells rather than markdown (or some combination of > both)? As soon as you put an underscore in the cell, everything until the > next underscore is rendered as oblique, which is not very easy to read. > > Someone would have to write a markdown+latex mode in code mirror that > detect $/$$ et switch to latex. > It would have the advantage of also highlighting latex correctly. > But it need to be done. > > is not hard, but not trivial either. > ? > M > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: