From thecolourblue+m at gmail.com Fri Mar 1 11:37:43 2013 From: thecolourblue+m at gmail.com (tcb) Date: Fri, 1 Mar 2013 16:37:43 +0000 Subject: [IPython-dev] ipython with twisted reactor Message-ID: Hi, I am trying to use the ipython notebook with some example code which sets up a websocket server (using autobahn python http://autobahn.ws/python/tutorials/echo ). The problem is that I need to call twisted's reactor.run() method, but this blocks all input, and I need to interact with the objects to send/receive messages. I haven't been able to get this to work in the notebook yet. I assume its a problem interacting with the tornado eventloop in ipython- but it could be something else (the examples work fine outside of ipython). Rather than calling reactor.run()- I've tried putting it into a thread: threading.Thread(target=reactor.run, kwargs={'installSignalHandlers':False}).start() but that doesn't work either- it doesn't ever run. The threadedselector doesn't seem to run either (putting this at the top of my script): from twisted.internet import _threadedselect reactor = _threadedselect.install() There is a possibility to interleave the twisted eventloop with other event loops: from twisted.internet import reactor reactor.interleave(foreignEventLoopWakerFunction) but I have no idea which ipython function would serve as a 'waker' function. Has anyone managed to do this already in the notebook? -thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Fri Mar 1 13:16:10 2013 From: benjaminrk at gmail.com (Min RK) Date: Fri, 1 Mar 2013 10:16:10 -0800 Subject: [IPython-dev] ipython with twisted reactor In-Reply-To: References: Message-ID: <04B11B71-4B0A-4286-9344-C180AEE46204@gmail.com> I don't know what the waker function would be, but the event loop IPython uses is tornado, So that's what twisted needs to work with. -MinRK On Mar 1, 2013, at 8:37, tcb wrote: > Hi, > > I am trying to use the ipython notebook with some example code which sets up a websocket server (using autobahn python http://autobahn.ws/python/tutorials/echo ). > > The problem is that I need to call twisted's reactor.run() method, but this blocks all input, and I need to interact with the objects to send/receive messages. I haven't been able to get this to work in the notebook yet. I assume its a problem interacting with the tornado eventloop in ipython- but it could be something else (the examples work fine outside of ipython). > > Rather than calling reactor.run()- I've tried putting it into a thread: > > threading.Thread(target=reactor.run, kwargs={'installSignalHandlers':False}).start() > > but that doesn't work either- it doesn't ever run. > > The threadedselector doesn't seem to run either (putting this at the top of my script): > > from twisted.internet import _threadedselect > reactor = _threadedselect.install() > > There is a possibility to interleave the twisted eventloop with other event loops: > > from twisted.internet import reactor > > reactor.interleave(foreignEventLoopWakerFunction) > > but I have no idea which ipython function would serve as a 'waker' function. > > Has anyone managed to do this already in the notebook? > > -thanks > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From nborwankar at gmail.com Fri Mar 1 21:42:56 2013 From: nborwankar at gmail.com (Nitin Borwankar) Date: Fri, 1 Mar 2013 18:42:56 -0800 Subject: [IPython-dev] Hierarchical notebook listing In-Reply-To: <67358ECD-CFF9-4ED6-840D-9349F657EF32@gmail.com> References: <67358ECD-CFF9-4ED6-840D-9349F657EF32@gmail.com> Message-ID: Hi all, [[ this may sound like a horrible idea at first but it has legs and has been used in limited context ]] I think an "attribute" on the notebook called "accesspath" and an HTTP resolvable URI ( file:// or http:// or even mongo:// .... ) as a value makes this OS independent to a first order. It also allows the storage implementation code to be decoupled from the client in ways that do not require re-inventing these wheels and reuse of tons of built-in and well understood code. Also for implementations in othe languages ( Ruby...) the address remains invariant. Note: storage in SQL db's is probably best wrapped in a REST API for such purposes, with a set of CRUD primitives mapped to REST verbs, so we assume that SQL db's are behind an http:// or https:// scheme. Bottom line using existing resolvable HTTP URI's for storage addressing gives both the hierarchy and the decoupling in an IPyNB friendly metaphor without need to write Yet Another Platform Independent Layer ( YAPIL ? ) [[ long long sidebar: having such an address also allows storage of notebooks (JSON blobs) in say Apache CouchDB which is an awesome REST addressable JSON store, which also has very powerful master-master replication. CouchDB has an efficient storage format that saves embedded binaries and also allows custom indexes to be created using Map Reduce. Having your own notebooks and a co-workers notebooks and being able to sync them on a LAN is effortless, as is syncing them to the cloud. I have used Apache CouchDB (was the project manager for an NSF funded project in 2008-10) for JSON based bibliographies that could be synced. A notebook storage in CouchDB + bibliographies in CouchDB would be a very powerful researchers' publication management platform. In our project we had spec'ed out what an individual academic's bibliomanager, a departmental bibliomanager and a univ level biblio manager would look like. We had also talked to administrators in the UCBerkeley statistics department to investigate how it would improve their work. It was not implemented but another system which used CouchDB as a platform for creating citable URL's for unsolved Math problems was successfully implemented. It managed inline LaTex using the precursor to MathJax - JSMath - as MathJax did not exist then. We released on the 150th anniversary of the Riemann Hypothesis, in 2009, were Slashdotted and survived even when on a small EC2 instance - CouchDB is very resilient and manages concurrent requests very very well, in a resource efficient manner. P.S. I do not sell CouchDB :-) ]] ------------------------------------------------------------------ Nitin Borwankar nborwankar at gmail.com On Tue, Feb 26, 2013 at 2:47 AM, Matthias BUSSONNIER < bussonniermatthias at gmail.com> wrote: > > Le 26 f?vr. 2013 ? 01:49, Brian Granger a ?crit : > > > Some of the questions we need to think about: > > > > * What does the web service look like to move around and query > directories. > > * Do we map directory paths onto notebook URLS? If so, how? > > * How do we build a UI/UX that is extremely simple, but functional > > enough to get the job done. > > * How do we want to abstract these things for different notebook backend > stores? > > IMHO, the "backend" should return a notebook "document name" ( not the one > store in the json metadata) > and a UUID when asked to "list" notebook > > The actual content of the notebook when asked. > > The actual "mapping" filesystem/url would be using uuid that are generated > from hmac and path, so unique, and persistent. > This also work for database > | uuid | document_name | content | > > For the UI, > I can see "breadcrumbs" to navigate in parents directory. > We could also do something like github, where you show both ipynb and > folder and you can enter a folder. > > This would require some "custom" data structure that could be exchange > from dashboard and backend. > > -- > Matthias > > > > * How do notebook directories get mapped to kernel cwds? > > > Cheers, > > > > Brian > > > > On Mon, Feb 25, 2013 at 2:03 AM, Robert Young > wrote: > >> Hi, > >> > >> This email is intended to start a conversation around hierarchical > notebook > >> listings. I submitted a pull request [1] and it was pointed out that > >> supporting directories deserves some thought and discussion. > >> > >> Rob > >> > >> [1] https://github.com/ipython/ipython/pull/2977 > >> > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > > > > > > > -- > > Brian E. Granger > > Cal Poly State University, San Luis Obispo > > bgranger at calpoly.edu and ellisonbg at gmail.com > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nborwankar at gmail.com Fri Mar 1 22:00:19 2013 From: nborwankar at gmail.com (Nitin Borwankar) Date: Fri, 1 Mar 2013 19:00:19 -0800 Subject: [IPython-dev] Hierarchical notebook listing In-Reply-To: References: Message-ID: Hi Brian, [[ good chatting at Strata yesterday BTW ]] Pl. see other email which suggests just starting with resolvable HTTP URI as "accesspath". I hope this has not already been discussed and discarded - if so I will return from self-imposed embarrassment-exile, eventually. Some of the questions we need to think about: * What does the web service look like to move around and query directories. To a first approximation it looks like a web server with REST primitives that include PUT and DELETE as well as GET, POST. * Do we map directory paths onto notebook URLS? If so, how? ( we use a file:// scheme, for local paths and an http(s):// scheme for remote paths with a doc root defined at the target host. ) * How do we build a UI/UX that is extremely simple, but functional enough to get the job done. My weak attempt at this: - We don't build a general UI as permissions to remote stores may make matters complicated. Locally we delegate to the file browser, remotely we follow what looks like a directory listing when you browse say an Apache Webserver's directory that has read permissions. this is not how we implement it but it is a starting point for how it might look. * How do we want to abstract these things for different notebook backend stores? IMHO resolvable HTTP URI takes us most of the way - insist on a REST API for remote backends else you'll be navigating multiple levels of adapter abstraction hell. * How do notebook directories get mapped to kernel cwds? define an "IPyNB root" just like an HTTP document root "/" in a webserver. define it separately on the client, separately on the server/kernel machine. The HTTP URI is with respect to the root of whatever the HTTP target is. This also avoids exposing the actual dir structure and allows transparent changes of implementation, host machine, host OS ... *BEST* would be to define a standard REST interface for storage backends in a separate doc. Cheers, Brian ------------------------------------------------------------------ Nitin Borwankar nborwankar at gmail.com On Mon, Feb 25, 2013 at 4:49 PM, Brian Granger wrote: > Some of the questions we need to think about: > > * What does the web service look like to move around and query directories. > * Do we map directory paths onto notebook URLS? If so, how? > * How do we build a UI/UX that is extremely simple, but functional > enough to get the job done. > * How do we want to abstract these things for different notebook backend > stores? > * How do notebook directories get mapped to kernel cwds? > > Cheers, > > Brian > > On Mon, Feb 25, 2013 at 2:03 AM, Robert Young wrote: > > Hi, > > > > This email is intended to start a conversation around hierarchical > notebook > > listings. I submitted a pull request [1] and it was pointed out that > > supporting directories deserves some thought and discussion. > > > > Rob > > > > [1] https://github.com/ipython/ipython/pull/2977 > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > Brian E. Granger > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu and ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sat Mar 2 15:02:26 2013 From: benjaminrk at gmail.com (MinRK) Date: Sat, 2 Mar 2013 12:02:26 -0800 Subject: [IPython-dev] IPEP 12: Splitting KernelManager and KernelClient Message-ID: I've added IPEP 12 for detailing how KernelManager and KernelClient will be split into separate classes. There's still some figuring out to do in terms of module layout and heartbeat implementations, but it's a start. -MinRK -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sat Mar 2 15:30:09 2013 From: benjaminrk at gmail.com (MinRK) Date: Sat, 2 Mar 2013 12:30:09 -0800 Subject: [IPython-dev] IPEP 13 Message-ID: I've started IPEP 13, for the planned updates to the message spec. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthieu.brucher at gmail.com Mon Mar 4 05:37:53 2013 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Mon, 4 Mar 2013 10:37:53 +0000 Subject: [IPython-dev] Execnet Message-ID: Hi all, I'm sending this mail to both lists because I know that all of you work on parallel jobs/tasks/... Someone refered me to Execnet (http://codespeak.net/execnet/). Do you know of it and if you do, how does it compare to the solutions available in IPython or sklearn? Regards, Matthieu -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthieu.brucher at gmail.com Mon Mar 4 05:47:55 2013 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Mon, 4 Mar 2013 10:47:55 +0000 Subject: [IPython-dev] [Scikit-learn-general] Execnet In-Reply-To: References: Message-ID: I didn't check the license... Strange because it is supposed to be used in a commercial product soon. I will look into it a little bit further. Thanks for the feedback! Matthieu 2013/3/4 Olivier Grisel > 2013/3/4 Matthieu Brucher : > > Hi all, > > > > I'm sending this mail to both lists because I know that all of you work > on > > parallel jobs/tasks/... > > Someone refered me to Execnet (http://codespeak.net/execnet/). Do you > know > > of it and if you do, how does it compare to the solutions available in > > IPython or sklearn? > > Interesting, I did not know about it. Unfortunately the license is > GPL. Since would be interested to do write an optional joblib plugin > (we would also need a joblib plugin to use IPython.parallel cluster > BTW :) > > -- > Olivier > http://twitter.com/ogrisel - http://github.com/ogrisel > > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_d2d_feb > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general at lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > -- Information System Engineer, Ph.D. Blog: http://matt.eifelle.com LinkedIn: http://www.linkedin.com/in/matthieubrucher Music band: http://liliejay.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Mar 4 22:34:34 2013 From: benjaminrk at gmail.com (MinRK) Date: Mon, 4 Mar 2013 19:34:34 -0800 Subject: [IPython-dev] 0.13.2 Message-ID: I've started the process of backporting PRs for an 0.13.2 bugfix release, the results of which can be seen by looking at Issues labeled backport-0.13.2 . The following PRs have been backported so far: 2399 2503 2500 2498 2490 2491 2511 2294 2554 2561 2546 2526 2591 2582 2574 2638 2656 2663 2668 2662 2686 2685 2713 2717 2544 2749 2738 2769 2773 2778 2757 2783 2796 2799 2816 2830 2849 2859 2861 2863 2852 2864 2926 2924 2901 If there are any others you think should be applied for 0.13.2, or if any of those shouldn't be included, let me know. In general, these are bugfixes that are either small or important. No new features or APIs, etc. This is currently reflected in the 0.13.x branch . -MinRK -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave.hirschfeld at gmail.com Tue Mar 5 09:58:09 2013 From: dave.hirschfeld at gmail.com (Dave Hirschfeld) Date: Tue, 5 Mar 2013 14:58:09 +0000 (UTC) Subject: [IPython-dev] DictDB Message-ID: In my ipcontroller_config.py script I've got the following line: c.HubFactory.db_class = 'IPython.parallel.controller.dictdb.DictDB' I noticed in the log lots of messages about results being culled and found that the DictDB has `size_limit`, `record_limit` and `cull_fraction` attributes https://github.com/ipython/ipython/blob/master/IPython/parallel/controller/dictd b.py#L92 There doesn't seem to be any mention of them in the config file so I was wondering how to set these values. I assumed it might be something like c.HubFactory.db_class.size_limit = 10737418240 c.HubFactory.db_class.record_limit = ... c.HubFactory.db_class.cull_fraction = ... ..but was wondering how I could verify if these settings had worked - e.g. is there a way to query the controller settings? Thanks, Dave NB: Is the default 1MB or am I reading it wrong? size_limit = Integer(1024*1024, config=True, help="""The maximum total size (in bytes) of the buffers stored in the db From benjaminrk at gmail.com Tue Mar 5 12:26:49 2013 From: benjaminrk at gmail.com (MinRK) Date: Tue, 5 Mar 2013 09:26:49 -0800 Subject: [IPython-dev] DictDB In-Reply-To: References: Message-ID: On Tue, Mar 5, 2013 at 6:58 AM, Dave Hirschfeld wrote: > In my ipcontroller_config.py script I've got the following line: > > c.HubFactory.db_class = 'IPython.parallel.controller.dictdb.DictDB' > > I noticed in the log lots of messages about results being culled and > found that the DictDB has `size_limit`, `record_limit` and `cull_fraction` > attributes > > > https://github.com/ipython/ipython/blob/master/IPython/parallel/controller/dictd > b.py#L92 > > There doesn't seem to be any mention of them in the config file so I was > wondering how to set these values. > > I assumed it might be something like > > c.HubFactory.db_class.size_limit = 10737418240 > c.HubFactory.db_class.record_limit = ... > c.HubFactory.db_class.cull_fraction = ... > > > ..but was wondering how I could verify if these settings had worked - e.g. > is > there a way to query the controller settings? > There is not a way to query the controller's state, but this is not quite how config works. The way to think about config is setting class attributes, so to set the size limit of a DictDB object, you would do: c.DictDB.size_limit = 2**30 There is not instance-like notion where the name might change, depending on where it is assigned, it is always keyed by the name of the configurable class. > Thanks, > Dave > > NB: Is the default 1MB or am I reading it wrong? > This was a dumb typo, I wonder why it hasn't been caught before. Both DictDB's exclusion from help output and the default value should be fixed by: https://github.com/ipython/ipython/pull/2997 > > size_limit = Integer(1024*1024, config=True, > help="""The maximum total size (in bytes) of the buffers stored in the db > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Tue Mar 5 12:26:49 2013 From: benjaminrk at gmail.com (MinRK) Date: Tue, 5 Mar 2013 09:26:49 -0800 Subject: [IPython-dev] DictDB In-Reply-To: References: Message-ID: On Tue, Mar 5, 2013 at 6:58 AM, Dave Hirschfeld wrote: > In my ipcontroller_config.py script I've got the following line: > > c.HubFactory.db_class = 'IPython.parallel.controller.dictdb.DictDB' > > I noticed in the log lots of messages about results being culled and > found that the DictDB has `size_limit`, `record_limit` and `cull_fraction` > attributes > > > https://github.com/ipython/ipython/blob/master/IPython/parallel/controller/dictd > b.py#L92 > > There doesn't seem to be any mention of them in the config file so I was > wondering how to set these values. > > I assumed it might be something like > > c.HubFactory.db_class.size_limit = 10737418240 > c.HubFactory.db_class.record_limit = ... > c.HubFactory.db_class.cull_fraction = ... > > > ..but was wondering how I could verify if these settings had worked - e.g. > is > there a way to query the controller settings? > There is not a way to query the controller's state, but this is not quite how config works. The way to think about config is setting class attributes, so to set the size limit of a DictDB object, you would do: c.DictDB.size_limit = 2**30 There is not instance-like notion where the name might change, depending on where it is assigned, it is always keyed by the name of the configurable class. > Thanks, > Dave > > NB: Is the default 1MB or am I reading it wrong? > This was a dumb typo, I wonder why it hasn't been caught before. Both DictDB's exclusion from help output and the default value should be fixed by: https://github.com/ipython/ipython/pull/2997 > > size_limit = Integer(1024*1024, config=True, > help="""The maximum total size (in bytes) of the buffers stored in the db > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Mar 6 07:16:34 2013 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 6 Mar 2013 12:16:34 +0000 Subject: [IPython-dev] IPyTables - simple table construction for IPython Message-ID: Working in the Notebook recently, I've found that I often want to display results in a table. pandas' DataFrames display nicely, but they're not terribly convenient if your data isn't already in that form, and I found myself assembling HTML in my code. Enter IPyTables. It's a simple interface to build tables for display in the notebook. At the easiest, you just pass in a series of tuples: Table((4, 1, 8), (9, 7, 3), (5, 2, 6)) With a little bit more effort, you can add header cells and customise the formatting. Demo: http://nbviewer.ipython.org/5098827 Module: https://gist.github.com/takluyver/5098835 This is a prototype that I've thrown together quickly: feedback is very welcome. Obvious extensions include: - Expose more style attributes for customisation (so far it's just text and background colour) - Add a plain text repr(), so that tables are useable in the terminal Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason-sage at creativetrax.com Wed Mar 6 10:18:55 2013 From: jason-sage at creativetrax.com (Jason Grout) Date: Wed, 06 Mar 2013 09:18:55 -0600 Subject: [IPython-dev] IPyTables - simple table construction for IPython In-Reply-To: References: Message-ID: <51375E5F.3010801@creativetrax.com> On 3/6/13 6:16 AM, Thomas Kluyver wrote: > Working in the Notebook recently, I've found that I often want to > display results in a table. pandas' DataFrames display nicely, but > they're not terribly convenient if your data isn't already in that form, > and I found myself assembling HTML in my code. > > Enter IPyTables. It's a simple interface to build tables for display in > the notebook. At the easiest, you just pass in a series of tuples: > > Table((4, 1, 8), > (9, 7, 3), > (5, 2, 6)) > > With a little bit more effort, you can add header cells and customise > the formatting. > > Demo: http://nbviewer.ipython.org/5098827 > Module: https://gist.github.com/takluyver/5098835 > > This is a prototype that I've thrown together quickly: feedback is very > welcome. Obvious extensions include: > > - Expose more style attributes for customisation (so far it's just text > and background colour) > - Add a plain text repr(), so that tables are useable in the terminal > Cool! Another cool thing would be a cell decorator that let you paste in csv data or space-separated data: %%table 1 2 3 4 5 6 7 8 9 or %%table 1,2,3 4,5,6 7,8,9 Especially if somehow that list of lists could be assigned to a variable. Thanks, Jason From sunset.smile at gmail.com Wed Mar 6 13:12:48 2013 From: sunset.smile at gmail.com (Saeed Moghimi) Date: Wed, 6 Mar 2013 10:12:48 -0800 Subject: [IPython-dev] Toggle code Message-ID: Hi, I just would like to know if Toggle the code and mark down cells also could be implemented or might be already there. specially when presenting some thing some times large block of code is disturbing. Regards, Saeed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Wed Mar 6 13:25:17 2013 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 6 Mar 2013 10:25:17 -0800 Subject: [IPython-dev] Toggle code In-Reply-To: References: Message-ID: We have decided to not implement this in IPython itself, but it is trivial to do yourself in JavaScript. A bit of Googling will show some more examples, but here is one: http://ganwellresource.blogspot.co.uk/2012/11/hide-ipython-notebook-input-boxes.html On Wed, Mar 6, 2013 at 10:12 AM, Saeed Moghimi wrote: > Hi, > I just would like to know if Toggle the code and mark down cells also could > be implemented or might be already there. specially when presenting some > thing some times large block of code is disturbing. > Regards, > Saeed. > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From ellisonbg at gmail.com Wed Mar 6 14:37:09 2013 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 6 Mar 2013 11:37:09 -0800 Subject: [IPython-dev] [IPython-User] ANN: Exhibitionist - integrating HTML/JS UIs into interactive python work In-Reply-To: <20130306122211.150060@gmx.com> References: <20130306122211.150060@gmx.com> Message-ID: This is really cool to see work like this happening. Very nice! I wanted to update you on the development situation with IPython that may affect your code: * We plan on starting to work on creating a nice architecture for interactive JavaScript widgets, in late summer. * This architecture will enable all of this to be done without any additional server logic. * Using additional server logic as you have done is not officially supported. What I mean by this is that the notebook architecture may change in a way that makes it impossible to do this type of thing. The problem is that you have made some strong assumptions about where the notebook server, kernel and your server are running. The preferred way of getting data back and forth between python and the broser is to use our message channels. Currently these channels are not sufficient for what you want to do, but after are work later this year, they will be. * Because of security issues are are moving away from the notebook being able to execute dynamically generated javascript code. We will replace this with a javascript plugin system that is more secure. You will have to rewrite things when this happens. Cheers, Brian On Wed, Mar 6, 2013 at 4:22 AM, yoval p. wrote: > Hi Everyone, > > > I'm pleased to announce my new project, now available on Github, > called Exhibitionist, a library geared towards integrating HTML/JS > UIs into interactive python work. > > There's a lot of experimentation going on in this space, This is my > offering. > The proposed solutions I've seen so far all seem to stray in one way or > another > from the beaten web-development track. Exhibitionist tries to do things > as closely as possible to the way they generally work in web development. > > A short summary of the main points: > > - data is exchanged using AJAX, your python objects become API > endpoints on an in-process web server. > - you can exchange messages between python and javascript, in either > direction, using websockets and a built-in PubSub implementation. > - no quoted code. > that means, no "console.log('def myfunc():\nprint \"it\"'), > nor 's += "console.log(\"you are quoting again...\")"' > > javascript lives in js files, python in .py, and HTML can be generated > with your favorite template engine. > - Views are vanilla web application viewable in any browser, no IPython > dependency for use or development. > - OTOH why wouldn't you want to use IPython-notebook's awesome inline HTML > display?, > works just like you'd expect. > - Repo contains serveral examples, including a heavily documented skeleton > project > - Supported Python 2.6,2.7,3.1,3.2 > - Tests, Coverage, tox, travis, yep. > - BSD licensed > - Currently Tested on linux only, you can help there. > > The primary example included is a javascript grid UI for pandas Dataframes > that loads data on-demand via AJAX. a screenshot is available from the > Github > project page. > > Exhibitionist is built as a standalone library on top of Tornado (also > used by IPython), and it allows you to craft views either for 3rd party > library objects, but I also Imagine it as a primary dependency of libraries > in the pydata arena, that integrate HTML views as core functionality of > the library, allowing all it's users to work with rich views either > in a modern browser, or in IPython-notebook for the best UX. > > Repo lives at: https://github.com/Exhibitionist/Exhibitionist > > Please try it out, report any issues you find and provide feedback. > I hope you find it useful. > > _______________________________________________ > IPython-User mailing list > IPython-User at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-user > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From fperez.net at gmail.com Wed Mar 6 16:07:54 2013 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 6 Mar 2013 13:07:54 -0800 Subject: [IPython-dev] [IPython-User] ANN: Exhibitionist - integrating HTML/JS UIs into interactive python work In-Reply-To: References: <20130306122211.150060@gmx.com> Message-ID: Hey Brian, On Wed, Mar 6, 2013 at 11:37 AM, Brian Granger wrote: > This is really cool to see work like this happening. Very nice! > > I wanted to update you on the development situation with IPython that > may affect your code: [...] I was just thinking, it might be worth adding some of this language to the wiki's roadmap. There's a lot of energy bubbling up from the community right now on this topic, so having our plans visible to the public beyond the mailing list is probably worthwhile. That way people will more easily know what to plan for and where we're headed, so they don't spend too much time on building infrastructure that we might make obsolete soon. Cheers, f From ellisonbg at gmail.com Wed Mar 6 16:45:30 2013 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 6 Mar 2013 13:45:30 -0800 Subject: [IPython-dev] [IPython-User] ANN: Exhibitionist - integrating HTML/JS UIs into interactive python work In-Reply-To: References: <20130306122211.150060@gmx.com> Message-ID: Done...thanks for pointing this out... Brian On Wed, Mar 6, 2013 at 1:07 PM, Fernando Perez wrote: > Hey Brian, > > > On Wed, Mar 6, 2013 at 11:37 AM, Brian Granger wrote: >> This is really cool to see work like this happening. Very nice! >> >> I wanted to update you on the development situation with IPython that >> may affect your code: > > [...] > > I was just thinking, it might be worth adding some of this language to > the wiki's roadmap. There's a lot of energy bubbling up from the > community right now on this topic, so having our plans visible to the > public beyond the mailing list is probably worthwhile. That way > people will more easily know what to plan for and where we're headed, > so they don't spend too much time on building infrastructure that we > might make obsolete soon. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From y-p at gmx.com Wed Mar 6 17:36:10 2013 From: y-p at gmx.com (yoval p.) Date: Wed, 06 Mar 2013 23:36:10 +0100 Subject: [IPython-dev] ANN: Exhibitionist - integrating HTML/JS UIs into interactive python work Message-ID: <20130306223610.150100@gmx.com> Here's hoping mailman picks this up into the correct thread... Hi Brian/Matthaias, Thanks for the heads up. I'm looking forward to superior functionality being available in IPython, though your roadmap is unclear on whether this will actually land late this year or in 2014. To reiterate, The only IPython functionality needed is the ability to load and view remote HTML pages, and that only as a substitute to using a separate browser window. Is that really something you're planning to disallow? If so, I strongly urge you not to throw away a feature that makes IPython-notebook as powerful as the web. Wouldn't an Opt-in mechanism be sufficient? As for strong assumptions, The only assumption I'm really making is that there's a network connection between the front-end and where the kernel is. Cross-origin issues are already addressed with built-in JSONP and CORS support. Could you elaborate on how this assumption might be violated in the future? Surely a user working on his laptop on a local file, with a local kernel should be able to access local urls. I'm less interested in other use-cases, and frankly don't have the resources to pursue them. I'm trying to keep things generic, so hopefully views will be easily portable to the new architecture when it arrives. I've started work on a crossfilter (http://square.github.com/crossfilter/ ) based "lasso" UI, that allows you to graphically filter a python dataset by selecting points across different dimensions, and pick up the filtered dataset back in python. Unless I hit a wall, there should be a demo up in a couple of weeks. stay tuned, Yoval Matthaias Bussonnier Wrote: > Do you rely on display_javascript for the initial loading of javascript ? or inject