From fperez.net at gmail.com Sat Mar 1 18:19:11 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 1 Mar 2008 15:19:11 -0800 Subject: [IPython-dev] ipython1 on launchpad Message-ID: Hi all, we're here at Enthought working on various ipython/scipy/sage related things, and we'll begin testing Launchpad/bzr for the ipython1 work now. In a few weeks of use for ipython1 this will give us a good amount of real world experience with bzr, and in the meantime Enthought will look at their setup and possibilities regarding hg hosting with migration of the existing Trac database. After the Paris sprint (March 22-23), we can look into either starting to use launchpad/bzr or Enthought/hg/trac, based on the experience from all of us (Ville and Brian had already started using Launchpad a few days ago). If we go with Launchpad, we'll need to look into a way of migrating the existing Trac database, if possible. It would be nice to have closer integration between the bug tracking and the code, though for now we'll continue to use Trac as is. So bear with us for a few weeks while the code is a bit all over the place. For now, we'll continue to update the regular SVN trunk and ipython1 branches at ipython.scipy.org, so that those following SVN don't need to track our madness. Cheers, f From glenn at tarbox.org Sat Mar 1 23:55:27 2008 From: glenn at tarbox.org (Glenn Tarbox, PhD) Date: Sat, 1 Mar 2008 20:55:27 -0800 Subject: [IPython-dev] Twisted Qt4 integration... Message-ID: As IPython1 uses Twisted and Matplotlib supports Qt, this may be of interest. I've got a somewhat functional Qt4 reactor working. Its maintained and available at: http://code.tarbox.org/qtreactor There is git-daemon and git-instaweb access. git-instaweb will auto-generate tarballs for download. I'm supporting and hosting this code as there are issues arising from the clash between Twisted's MIT license and Qt's GPL. -glenn -- Glenn H. Tarbox, PhD glenn at tarbox.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From vivainio at gmail.com Sun Mar 2 05:19:52 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sun, 2 Mar 2008 12:19:52 +0200 Subject: [IPython-dev] ipython1 on launchpad In-Reply-To: References: Message-ID: <46cb515a0803020219y4699cab2vaee5db98331b80b1@mail.gmail.com> On Sun, Mar 2, 2008 at 1:19 AM, Fernando Perez wrote: > we're here at Enthought working on various ipython/scipy/sage related > things, and we'll begin testing Launchpad/bzr for the ipython1 work > now. In a few weeks of use for ipython1 this will give us a good > amount of real world experience with bzr, and in the meantime > Enthought will look at their setup and possibilities regarding hg > hosting with migration of the existing Trac database. In my observation, bzr and launchpad are just fine, and I don't think you guys will have any problems with it either. Performance is good, and the merging I have done so far has worked wihouth a hitch. At this point I'm not even sure whether hg really *is* better than bzr, now that early bzr performance problems don't exist anymore. hg has slightly better GUI support on windows, though, but I don't really care about that. Summary summarum, I am not sure whether setting up own servers (in order to host hg) is worth the time and expense, and the results could be worse (for example, would it be possible for *anyone* to register a public branch on the custom server, like with Launchpad?) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Sun Mar 2 12:17:21 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 2 Mar 2008 09:17:21 -0800 Subject: [IPython-dev] ipython1 on launchpad In-Reply-To: <46cb515a0803020219y4699cab2vaee5db98331b80b1@mail.gmail.com> References: <46cb515a0803020219y4699cab2vaee5db98331b80b1@mail.gmail.com> Message-ID: On Sun, Mar 2, 2008 at 2:19 AM, Ville M. Vainio wrote: > Summary summarum, I am not sure whether setting up own servers (in > order to host hg) is worth the time and expense, and the results could > be worse (for example, would it be possible for *anyone* to register a > public branch on the custom server, like with Launchpad?) That's pretty much my take so far on the problem, after looking at Launchpad a lot yesterday. We'll just wait for a few weeks (we have another 2 sprints coming up in March, one on scipy/neuroimaging and one on ipython in Paris) and sort out the final decision, but so far from what I see, I've reached the same conclusion as above. Launchpad has a lot of good infrastructure, and I doubt it's worth Enthought's time to redo it. If we go that route, it would be great if we could find a way to migrate our Trac history into their bug system so we keep our tickets. Cheers, f From gael.varoquaux at normalesup.org Fri Mar 7 14:10:36 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 7 Mar 2008 20:10:36 +0100 Subject: [IPython-dev] Parallel map Message-ID: <20080307191036.GK14012@phare.normalesup.org> Hi, I am trying to do a parallel map using ipython1. Is there a really simple way to do this, or a tutorial somewhere telling me how? I can probably figure it out, but I have to dig through a fair amount of tutorial/doc/wiki articles/reading source code to move forward. My requirement is that I want the code to be purely valid self sustained Python code. IMHO ipython1 should provide a few helper functions, including this one: a parallel, blocking, map. Cheers, Ga?l From gael.varoquaux at normalesup.org Fri Mar 7 14:42:26 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 7 Mar 2008 20:42:26 +0100 Subject: [IPython-dev] Parallel map In-Reply-To: <20080307191036.GK14012@phare.normalesup.org> References: <20080307191036.GK14012@phare.normalesup.org> Message-ID: <20080307194226.GM14012@phare.normalesup.org> On Fri, Mar 07, 2008 at 08:10:36PM +0100, Gael Varoquaux wrote: > I am trying to do a parallel map using ipython1. Is there a really simple > way to do this, or a tutorial somewhere telling me how? I can probably > figure it out, but I have to dig through a fair amount of > tutorial/doc/wiki articles/reading source code to move forward. > My requirement is that I want the code to be purely valid self sustained > Python code. OK, making some progress at this. I found out I need to create a MultiEngineClient rc = client.MultiEngineClient(('127.0.0.1', 10105)) and I can use its map method. My problem is that I do not know how to create the engines from Python. I guess I will have to RTFS ipcontroller. Ga?l From vivainio at gmail.com Sat Mar 8 02:07:29 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sat, 8 Mar 2008 09:07:29 +0200 Subject: [IPython-dev] Bzr plugin to check line endings Message-ID: <46cb515a0803072307q3f3bfee6m77d3ab30f153c2bc@mail.gmail.com> There now exist a plugin (pre-commit-hook) for bzr that prevents us from accidentally checking in code with CRLF line endings. Thank you Alexander! I've added .checkeol to the root of stable-dev, it will work if you install the following plugin to your bazaar plugins directory: http://bazaar.launchpad.net/~bialix/+junk/checkeol For my Vista machine, the plugins directory is: C:\Users\ville\AppData\Roaming\bazaar\2.0\plugins (Gotta love those windows directories...) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From gael.varoquaux at normalesup.org Sat Mar 8 05:03:29 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sat, 8 Mar 2008 11:03:29 +0100 Subject: [IPython-dev] Parallel map In-Reply-To: <20080307194226.GM14012@phare.normalesup.org> References: <20080307191036.GK14012@phare.normalesup.org> <20080307194226.GM14012@phare.normalesup.org> Message-ID: <20080308100329.GA22261@phare.normalesup.org> On Fri, Mar 07, 2008 at 08:42:26PM +0100, Gael Varoquaux wrote: > On Fri, Mar 07, 2008 at 08:10:36PM +0100, Gael Varoquaux wrote: > > I am trying to do a parallel map using ipython1. Is there a really simple > > way to do this, or a tutorial somewhere telling me how? I can probably > > figure it out, but I have to dig through a fair amount of > > tutorial/doc/wiki articles/reading source code to move forward. > > My requirement is that I want the code to be purely valid self sustained > > Python code. > OK, making some progress at this. > I found out I need to create a MultiEngineClient > rc = client.MultiEngineClient(('127.0.0.1', 10105)) > and I can use its map method. I succeeded (I had a good night(s sleep, in between), by piggy backing the ipcluster script. It is a bit ugly, but I post the code here for future reference. What made my task hard was both the fact that there is no obvious way of creating a cluster from Python, and the fact that ipython1.kernel.api was suppressed and that all the information I can find on the web uses ipython1.kerenl.api.RemoteControler. Now the irony is that I ended up not beeing able to use ipython1 for the problem I was interested in, as the objects I wanted to send to my parallel map where not picklable. I wrote a small hack using threading and os.system to do the work. I suspect this is a limitation people are going to bump into quite often. Ideas to make a workaround more or less part of ipython1 natively would be great. In my case, the object I had to scatter where directly imported from a module, so scattering a module path as a string (eg 'ipython1.kernel.client.MultiEngineClient') waould have been an option. I have no hindsight on these problems, so I don't pretend suggesting a good solution. Anyway, thanks for ipython1, keep the good work up, it is a difficult but import task, Cheers, Ga?l ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ """ Provides a simple parallel map. """ # Piggy-back the ipcluster script to start the engines. from ipython1.kernel.scripts import ipcluster as cluster from ipython1.kernel.client import MultiEngineClient from threading import Thread from time import sleep import sys ############################################################################## def guess_ncpu(): """ Parses /proc/cpuinfo to guess the number of CPU on the box. This has been tested only under Linux. """ ncpu = 0 cpuinfo = file('/proc/cpuinfo') for line in cpuinfo.readlines(): if line[:10] == 'processor\t': ncpu += 1 return ncpu ############################################################################## # Code to start the engine and create the controller def start_cluster(ncpu=guess_ncpu()): """ Starts a cluster on the local computer and returns a controller to the cluster. """ # We use ipcluster.main, but it takes its instructions from sys.argv, # thus we overide it orig_argv = sys.argv sys.argv = ['foo', '-n', str(ncpu)] # Starting the cluster is a blocking operation. We thus need a # thread to do the work. Thread(target=cluster.main).start() # There is a sleep(3) in ipcluster sleep(4) sys.argv = orig_argv return MultiEngineClient(('127.0.0.1',10105)) ############################################################################## # This code is so trivial you should really use directly the controller # method if you are going to do anything more than running pmap once # (keep in mind that there is an overhead of creating the cluster). def pmap(func, seq, ncpu=guess_ncpu()): """ Creates a cluster of ipython1 engines and runs a parallel map on it. """ mec = start_cluster(ncpu=ncpu) outseq = mec.map(func, seq) mec.kill(controler=True) return outseq ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ From fperez.net at gmail.com Sat Mar 8 05:28:02 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 8 Mar 2008 02:28:02 -0800 Subject: [IPython-dev] Parallel map In-Reply-To: <20080308100329.GA22261@phare.normalesup.org> References: <20080307191036.GK14012@phare.normalesup.org> <20080307194226.GM14012@phare.normalesup.org> <20080308100329.GA22261@phare.normalesup.org> Message-ID: On Sat, Mar 8, 2008 at 2:03 AM, Gael Varoquaux wrote: > I succeeded (I had a good night(s sleep, in between), by piggy backing > the ipcluster script. It is a bit ugly, but I post the code here for > future reference. > > What made my task hard was both the fact that there is no obvious way of > creating a cluster from Python, and the fact that ipython1.kernel.api was > suppressed and that all the information I can find on the web uses > ipython1.kerenl.api.RemoteControler. We need to summarize the recent work done at the sage/scipy/ipython sprint. In particular, Min did a lot of excellent work *precisely* on this issue, most (if not all) of which is already committed, to provide a full ipython daemon for process control. This allows you to do exactly that, to create/control/destroy engines and/or controllers from within python scripts. The good thing is that you're already a bzr launchpad team member, so I'm sure you'll soon be contributing this code and docs :) > Now the irony is that I ended up not beeing able to use ipython1 for the > problem I was interested in, as the objects I wanted to send to my > parallel map where not picklable. I wrote a small hack using threading > and os.system to do the work. I suspect this is a limitation people are > going to bump into quite often. Ideas to make a workaround more or less > part of ipython1 natively would be great. In my case, the object I had to > scatter where directly imported from a module, so scattering a module > path as a string (eg 'ipython1.kernel.client.MultiEngineClient') waould > have been an option. I have no hindsight on these problems, so I don't > pretend suggesting a good solution. We finally figured out a trick using the 'with' statement to allow you to write code like with all_engines: do_in_remote_engines(..) or with task_controller_using_engines(1,3,5): do_this() and_that() etc... so that the control scripts can read much more naturally. This is something you may recall Eric Jones and I started thinking about at Scipy'07, and finally on the plane back from Sage/Scipy Days 8 I was able to understand how it needed to be structured. That code isn't ready for inclusion anywhere yet, but the proof of concept is in my local copy and hopefully at the Paris sprint we can test it a bit and put it in. Being able to write pure-python (instead of the ugly code-inside-strings) parallel code will make the experience much more natural, I think, especially coupled with the new facilities for pushing function objects and the daemon-based ability to do clean multi-process interactive control. The pieces are falling in place, now we just need a breather to document all this and gradually communicate things better to everyone. I apologize if the process has been a bit opaque lately, but it's been just a very hectic time for me. After the Paris IPython sprint (March 22-23) I hope to catch my breath a bit, sort out the distributed VCS situation (bzr/launchpad is looking pretty good so far), and communicate better a set of clear plans so that all those who so kindly expressed interest and willingness to pitch in can do so with an understanding of where the various pieces of the puzzle are. Cheers, f From bialix at ukr.net Sat Mar 8 05:30:13 2008 From: bialix at ukr.net (Alexander Belchenko) Date: Sat, 08 Mar 2008 12:30:13 +0200 Subject: [IPython-dev] Bzr plugin to check line endings In-Reply-To: <46cb515a0803072307q3f3bfee6m77d3ab30f153c2bc@mail.gmail.com> References: <46cb515a0803072307q3f3bfee6m77d3ab30f153c2bc@mail.gmail.com> Message-ID: <47D26AB5.3000306@ukr.net> Ville M. Vainio ?????: > There now exist a plugin (pre-commit-hook) for bzr that prevents us > from accidentally checking in code with CRLF line endings. Thank you > Alexander! > > I've added .checkeol to the root of stable-dev, it will work if you > install the following plugin to your bazaar plugins directory: > > http://bazaar.launchpad.net/~bialix/+junk/checkeol > > For my Vista machine, the plugins directory is: > > C:\Users\ville\AppData\Roaming\bazaar\2.0\plugins > > (Gotta love those windows directories...) > You also can put it to C:\Program Files\Bazaar\plugins directory. It's a bit shorter ;-) From gael.varoquaux at normalesup.org Sat Mar 8 05:41:13 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sat, 8 Mar 2008 11:41:13 +0100 Subject: [IPython-dev] Parallel map In-Reply-To: References: <20080307191036.GK14012@phare.normalesup.org> <20080307194226.GM14012@phare.normalesup.org> <20080308100329.GA22261@phare.normalesup.org> Message-ID: <20080308104113.GC22261@phare.normalesup.org> On Sat, Mar 08, 2008 at 02:28:02AM -0800, Fernando Perez wrote: > We need to summarize the recent work done at the sage/scipy/ipython > sprint. I did see some recent commits on Trac :->. > In particular, Min did a lot of excellent work *precisely* on > this issue, most (if not all) of which is already committed, to > provide a full ipython daemon for process control. This allows you to > do exactly that, to create/control/destroy engines and/or controllers > from within python scripts. Great. Now a very important question: can the daemon be started from Python without an os.system apparent for the user? I want to be able to log in a box, and run a pPthon script that does parallel computing with only a few lines of Python code. > The good thing is that you're already a bzr launchpad team member, so > I'm sure you'll soon be contributing this code and docs :) Hum. Currently I am physicaly killing myself doing way to many things. No, seriously, its worst than it has ever been, and I have a dangerously off balance sleep pattern. Maybe I'll get better next year when I start my new job. Maybe I'll even use ipython1 as a full part of my day work. > We finally figured out a trick using the 'with' statement to allow you > to write code like > with all_engines: > do_in_remote_engines(..) That's really nice. Now a big question is, does that remove the need for all the objects in the parallel code to be picklable. Because that is what was limiting me here. Cheers, Ga?l From fperez.net at gmail.com Sat Mar 8 13:32:26 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 8 Mar 2008 10:32:26 -0800 Subject: [IPython-dev] Parallel map In-Reply-To: <20080308103958.GB22261@phare.normalesup.org> References: <20080307191036.GK14012@phare.normalesup.org> <20080307194226.GM14012@phare.normalesup.org> <20080308100329.GA22261@phare.normalesup.org> <20080308103958.GB22261@phare.normalesup.org> Message-ID: Hey, On Sat, Mar 8, 2008 at 2:39 AM, Gael Varoquaux wrote: > On Sat, Mar 08, 2008 at 02:28:02AM -0800, Fernando Perez wrote: > > We need to summarize the recent work done at the sage/scipy/ipython > > sprint. > > I did see some recent commits on Trac :->. > > > > In particular, Min did a lot of excellent work *precisely* on > > this issue, most (if not all) of which is already committed, to > > provide a full ipython daemon for process control. This allows you to > > do exactly that, to create/control/destroy engines and/or controllers > > from within python scripts. > > Great. Now a very important question: can the daemon be started from > Python without an os.system apparent for the user? I want to be able to > log in a box, and run a pPthon script that does parallel computing with > only a few lines of Python code. Yes, absolutely. That was a key requirement of the design, that we could provide fully self-contained examples of ipython parallel code that don't start by telling the user 'open 4 terminals and type this in each'... The docs will be catching up soon, likely after the Paris meeting. > > The good thing is that you're already a bzr launchpad team member, so > > I'm sure you'll soon be contributing this code and docs :) > > Hum. Currently I am physicaly killing myself doing way to many things. > No, seriously, its worst than it has ever been, and I have a dangerously > off balance sleep pattern. Maybe I'll get better next year when I start > my new job. Maybe I'll even use ipython1 as a full part of my day work. Well, don't worry: I made you a team member just so we can work in Paris with you fluidly, and to make it easy for you to prep anything you may need. I am NOT trying to bury you with more things, and I realize fullyt that ETS/M2 are higher priorities on this front than ipython. Please keep sleep and sanity above open source, really (I say this after having crashed hard with a migraine yesterday afternoon, probably from overdosing on work, stress and low sleep myself, so I really mean it when I say that I want your well-being to come first). > > We finally figured out a trick using the 'with' statement to allow you > > to write code like > > > with all_engines: > > do_in_remote_engines(..) > > That's really nice. Now a big question is, does that remove the need for > all the objects in the parallel code to be picklable. Because that is > what was limiting me here. Nope, pickle continues to be a requirement, except if you're sending data over MPI, case in which MPI does the communication. Cheers f From bialix at ukr.net Sun Mar 9 04:46:03 2008 From: bialix at ukr.net (Alexander Belchenko) Date: Sun, 09 Mar 2008 10:46:03 +0200 Subject: [IPython-dev] ipython1 on launchpad In-Reply-To: References: <46cb515a0803020219y4699cab2vaee5db98331b80b1@mail.gmail.com> Message-ID: <47D3A3CB.1040403@ukr.net> Fernando Perez ?????: > On Sun, Mar 2, 2008 at 2:19 AM, Ville M. Vainio wrote: > >> Summary summarum, I am not sure whether setting up own servers (in >> order to host hg) is worth the time and expense, and the results could >> be worse (for example, would it be possible for *anyone* to register a >> public branch on the custom server, like with Launchpad?) > > That's pretty much my take so far on the problem, after looking at > Launchpad a lot yesterday. We'll just wait for a few weeks (we have > another 2 sprints coming up in March, one on scipy/neuroimaging and > one on ipython in Paris) and sort out the final decision, but so far > from what I see, I've reached the same conclusion as above. Launchpad > has a lot of good infrastructure, and I doubt it's worth Enthought's > time to redo it. > > If we go that route, it would be great if we could find a way to > migrate our Trac history into their bug system so we keep our tickets. You need to talk with James Henstridge from Launchpad team about migrating of Trac tickets to Launchpad bug system. https://launchpad.net/~jamesh From fperez.net at gmail.com Sun Mar 9 04:53:22 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 9 Mar 2008 00:53:22 -0800 Subject: [IPython-dev] ipython1 on launchpad In-Reply-To: <47D3A3CB.1040403@ukr.net> References: <46cb515a0803020219y4699cab2vaee5db98331b80b1@mail.gmail.com> <47D3A3CB.1040403@ukr.net> Message-ID: On Sun, Mar 9, 2008 at 12:46 AM, Alexander Belchenko wrote: > > If we go that route, it would be great if we could find a way to > > migrate our Trac history into their bug system so we keep our tickets. > > You need to talk with James Henstridge from Launchpad team about > migrating of Trac tickets to Launchpad bug system. > > https://launchpad.net/~jamesh Thanks for the pointer. It will be a week or two (til after the next sprint) before we do this, but now we know who to talk to, that's great. Regards f From fperez.net at gmail.com Sun Mar 9 14:39:06 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 9 Mar 2008 11:39:06 -0700 Subject: [IPython-dev] bzr problems with attributes Message-ID: Hi Ville, since you started updating code in bzr already, I'm going to leave the back-merge into SVN trunk to you to avoid conflicts and will push bug fixes into bzr myself also. But I just noticed that the bzr branch was built without the proper execution attributes on all the files (things like the setup.py scripts aren't executable). Could you fix the bzr stable-dev branch? Thanks f From vivainio at gmail.com Sun Mar 9 14:45:57 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sun, 9 Mar 2008 20:45:57 +0200 Subject: [IPython-dev] bzr problems with attributes In-Reply-To: References: Message-ID: <46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com> On Sun, Mar 9, 2008 at 8:39 PM, Fernando Perez wrote: > since you started updating code in bzr already, I'm going to leave the > back-merge into SVN trunk to you to avoid conflicts and will push bug > fixes into bzr myself also. But I just noticed that the bzr branch > was built without the proper execution attributes on all the files > (things like the setup.py scripts aren't executable). Could you fix > the bzr stable-dev branch? Not really, I'm on windows :-) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Sun Mar 9 23:52:37 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 9 Mar 2008 20:52:37 -0700 Subject: [IPython-dev] bzr problems with attributes In-Reply-To: <46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com> References: <46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com> Message-ID: On Sun, Mar 9, 2008 at 11:45 AM, Ville M. Vainio wrote: > > On Sun, Mar 9, 2008 at 8:39 PM, Fernando Perez wrote: > > > since you started updating code in bzr already, I'm going to leave the > > back-merge into SVN trunk to you to avoid conflicts and will push bug > > fixes into bzr myself also. But I just noticed that the bzr branch > > was built without the proper execution attributes on all the files > > (things like the setup.py scripts aren't executable). Could you fix > > the bzr stable-dev branch? > > Not really, I'm on windows :-) OK, no problem. I'll fix it, but in the future if we need to make new public branches out of our tree, let me or another *nix person do it, so we don't have to manually back-fix all this (I know this started as a bzr experiment so I'm not blaming you or anyone else, just figuring out best practices from now on). Cheers, f From fperez.net at gmail.com Mon Mar 10 01:01:41 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 9 Mar 2008 22:01:41 -0700 Subject: [IPython-dev] bzr problems with attributes In-Reply-To: References: <46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com> Message-ID: On Sun, Mar 9, 2008 at 8:52 PM, Fernando Perez wrote: > On Sun, Mar 9, 2008 at 11:45 AM, Ville M. Vainio wrote: > > > > On Sun, Mar 9, 2008 at 8:39 PM, Fernando Perez wrote: > > > > > since you started updating code in bzr already, I'm going to leave the > > > back-merge into SVN trunk to you to avoid conflicts and will push bug > > > fixes into bzr myself also. But I just noticed that the bzr branch > > > was built without the proper execution attributes on all the files > > > (things like the setup.py scripts aren't executable). Could you fix > > > the bzr stable-dev branch? > > > > Not really, I'm on windows :-) > > OK, no problem. I'll fix it, but in the future if we need to make new > public branches out of our tree, let me or another *nix person do it, > so we don't have to manually back-fix all this (I know this started as > a bzr experiment so I'm not blaming you or anyone else, just figuring > out best practices from now on). OK, after a bit of find/grep this is now fixed on bzr. Cheers, f From fperez.net at gmail.com Mon Mar 10 01:13:14 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 9 Mar 2008 22:13:14 -0700 Subject: [IPython-dev] About IPython and bzr In-Reply-To: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> Message-ID: On Tue, Feb 19, 2008 at 10:46 AM, Ville M. Vainio wrote: > My recently-from-scratch installed IPython stable-dev (what used to be > called trunk/) is now very easy to check out from launchpad: > > bzr branch lp:ipython > > One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough. > Windows bzr does not support symlinks (even in a half-assed way like > svn), at least not yet, and I don't know whether such a checkin can be > reverted. So essentially, if there is even one checkin with symlink, I > would need to delete the branch and check it in again from local copy. Could you investigate these alternatives and let us know how well they work? http://bazaar-vcs.org/WindowsSymlinkSupport https://lists.ubuntu.com/archives/bazaar/2007q1/023801.html I'm asking because while we can live for now without symlinks on trunk, it looks like we'll be using bzr for everything in the future, and it's likely we may need to use symlinks for certain things. There are just too many things where symlinks are the right solution, and it doesn't make sense to avoid it just due to limitations of win32, so what we need is a good solution for win32 users. Alexander is on this list too, so perhaps he can provide us with useful feedback. Cheers, f From bialix at ukr.net Mon Mar 10 02:53:31 2008 From: bialix at ukr.net (Alexander Belchenko) Date: Mon, 10 Mar 2008 08:53:31 +0200 Subject: [IPython-dev] bzr problems with attributes In-Reply-To: <46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com> References: <46cb515a0803091145t4ea8c188ybe63070c1ef5b5fb@mail.gmail.com> Message-ID: <47D4DAEB.2060004@ukr.net> Ville M. Vainio ?????: > On Sun, Mar 9, 2008 at 8:39 PM, Fernando Perez wrote: > >> since you started updating code in bzr already, I'm going to leave the >> back-merge into SVN trunk to you to avoid conflicts and will push bug >> fixes into bzr myself also. But I just noticed that the bzr branch >> was built without the proper execution attributes on all the files >> (things like the setup.py scripts aren't executable). Could you fix >> the bzr stable-dev branch? > > Not really, I'm on windows :-) x-bit plugin is your friend ;-) From bialix at ukr.net Mon Mar 10 02:57:59 2008 From: bialix at ukr.net (Alexander Belchenko) Date: Mon, 10 Mar 2008 08:57:59 +0200 Subject: [IPython-dev] About IPython and bzr In-Reply-To: References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> Message-ID: <47D4DBF7.9040206@ukr.net> Fernando Perez ?????: > On Tue, Feb 19, 2008 at 10:46 AM, Ville M. Vainio wrote: >> My recently-from-scratch installed IPython stable-dev (what used to be >> called trunk/) is now very easy to check out from launchpad: >> >> bzr branch lp:ipython >> >> One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough. >> Windows bzr does not support symlinks (even in a half-assed way like >> svn), at least not yet, and I don't know whether such a checkin can be >> reverted. So essentially, if there is even one checkin with symlink, I >> would need to delete the branch and check it in again from local copy. > > Could you investigate these alternatives and let us know how well they work? > > http://bazaar-vcs.org/WindowsSymlinkSupport > https://lists.ubuntu.com/archives/bazaar/2007q1/023801.html > > I'm asking because while we can live for now without symlinks on > trunk, it looks like we'll be using bzr for everything in the future, > and it's likely we may need to use symlinks for certain things. > > There are just too many things where symlinks are the right solution, > and it doesn't make sense to avoid it just due to limitations of > win32, so what we need is a good solution for win32 users. > > Alexander is on this list too, so perhaps he can provide us with > useful feedback. I have special plugin win32symlinks that provide support for fake symlinks on Windows. At least checkout of branch with symlinks is possible without a hassle. It works for me very well. bzr branch lp:bzr-win32symlinks win32symlinks ;-) From vivainio at gmail.com Mon Mar 10 03:24:53 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 10 Mar 2008 09:24:53 +0200 Subject: [IPython-dev] About IPython and bzr In-Reply-To: <47D4DBF7.9040206@ukr.net> References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> <47D4DBF7.9040206@ukr.net> Message-ID: <46cb515a0803100024j774da6b7r28cbfb0e9a1da71f@mail.gmail.com> On Mon, Mar 10, 2008 at 8:57 AM, Alexander Belchenko wrote: > I have special plugin win32symlinks that provide support for fake symlinks on Windows. > At least checkout of branch with symlinks is possible without a hassle. > It works for me very well. Any reason why this is not in core? I did notice that there is a plugin available, but having to resort to a plugin when such a "core" functionality is at stake makes me feel a bit uneasy. Is it not stable yet, or not the "final answer"? Is there a danger it will botch history or merging? I still think we should at least try to avoid symlinks in bzr for a while more, until a definite solution settles down. And besides, if the symlink is the "right answer", it should be in an area that is not needed on Windows; obviously, even if you were able to check out the symlink, it would never actually work in win32. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Mon Mar 10 03:27:03 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 10 Mar 2008 00:27:03 -0700 Subject: [IPython-dev] About IPython and bzr In-Reply-To: <46cb515a0803100024j774da6b7r28cbfb0e9a1da71f@mail.gmail.com> References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> <47D4DBF7.9040206@ukr.net> <46cb515a0803100024j774da6b7r28cbfb0e9a1da71f@mail.gmail.com> Message-ID: On Mon, Mar 10, 2008 at 12:24 AM, Ville M. Vainio wrote: > I still think we should at least try to avoid symlinks in bzr for a > while more, until a definite solution settles down. And besides, if > the symlink is the "right answer", it should be in an area that is not > needed on Windows; obviously, even if you were able to check out the > symlink, it would never actually work in win32. I'm certainly not going to commit any symlinks for now, I just removed the ChangeLog one so I could use a local symlink on my system. But I was thinking of planning ahead, so that if we do need symlinks in the future for something, we know what the available solutions are. SVN didn't do it elegantly, but at least it (kinda) worked. Thanks to Alex for the info! Cheers, f From bialix at ukr.net Mon Mar 10 03:49:17 2008 From: bialix at ukr.net (Alexander Belchenko) Date: Mon, 10 Mar 2008 09:49:17 +0200 Subject: [IPython-dev] About IPython and bzr In-Reply-To: <46cb515a0803100024j774da6b7r28cbfb0e9a1da71f@mail.gmail.com> References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> <47D4DBF7.9040206@ukr.net> <46cb515a0803100024j774da6b7r28cbfb0e9a1da71f@mail.gmail.com> Message-ID: <47D4E7FD.8080608@ukr.net> Ville M. Vainio ?????: > On Mon, Mar 10, 2008 at 8:57 AM, Alexander Belchenko wrote: > >> I have special plugin win32symlinks that provide support for fake symlinks on Windows. >> At least checkout of branch with symlinks is possible without a hassle. >> It works for me very well. > > Any reason why this is not in core? I did notice that there is a > plugin available, but having to resort to a plugin when such a "core" > functionality is at stake makes me feel a bit uneasy. Is it not stable > yet, or not the "final answer"? Is there a danger it will botch > history or merging? No, only because of performance penalty. `bzr status` will pay extra time on additional stat call for every file. It costs about 15% overhead on NTFS filesystem and about 80% on FAT32. For merging fake symlinks approach into bzr core I need to rewrite standard os.listdir function to allow return not only file names, but stat info too. This should improve overall performance of walking across working tree even without fake symlinks support. So I'd like to call my plugin as "not final answer". > I still think we should at least try to avoid symlinks in bzr for a > while more, until a definite solution settles down. And besides, if > the symlink is the "right answer", it should be in an area that is not > needed on Windows; obviously, even if you were able to check out the > symlink, it would never actually work in win32. I'm not familiar with svn solution for symlinks on Windows, but my fake symlinks is not real symlinks, so you should not use it for pointing to some python package and expect that python import will work fine. But I follow symlinks format from Cygwin so at least it should appear as "real" symlinks in Cygwin environment. Details of this format could be found in plugin source. From vivainio at gmail.com Mon Mar 10 14:53:08 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 10 Mar 2008 20:53:08 +0200 Subject: [IPython-dev] Macros don't work in threaded shells Message-ID: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com> It appears recursive execution of _ip.runlines() is not possible with threaded shells (and so e.g. macros hang ipython) - noticed this through ticket #238, which is now a blocker. Try: ipython -q4thread > _ip.runlines('1') This will hang IPython. First, we come to MTInteractiveShell.runsource(""" _ip.runlines('1')""") This is given to runcode, which then does runsource( "1"). This never gets to runcode, because the thread is waiting for _ip.runlines('1') to complete. According to this comment, it seems this scenario has been considered, but the logic does not seem to work: # Case 3 # Store code in queue, so the execution thread can handle it. # Note that with macros and other applications, we MAY re-enter this # section, so we have to acquire the lock with non-blocking semantics, # else we deadlock. got_lock = self.thread_ready.acquire() self.code_queue.put(code) if got_lock: self.thread_ready.wait() # Wait until processed in timeout interval else: print "nolock",source self.thread_ready.release() (and no, using got_lock = self.thread_ready.acquire(blocking=False) won't fix it). Fernando? You remember whether this used to work? -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Mon Mar 10 15:17:08 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 10 Mar 2008 12:17:08 -0700 Subject: [IPython-dev] Macros don't work in threaded shells In-Reply-To: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com> References: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com> Message-ID: On Mon, Mar 10, 2008 at 11:53 AM, Ville M. Vainio wrote: > It appears recursive execution of _ip.runlines() is not possible with > threaded shells (and so e.g. macros hang ipython) - noticed this > through ticket #238, which is now a blocker. > > Try: > > ipython -q4thread > > > _ip.runlines('1') > > This will hang IPython. > > First, we come to MTInteractiveShell.runsource(""" _ip.runlines('1')""") > > This is given to runcode, which then does runsource( "1"). This never > gets to runcode, because the thread is waiting for _ip.runlines('1') > to complete. > > According to this comment, it seems this scenario has been considered, > but the logic does not seem to work: > > # Case 3 > # Store code in queue, so the execution thread can handle it. > > # Note that with macros and other applications, we MAY re-enter this > # section, so we have to acquire the lock with non-blocking semantics, > # else we deadlock. > got_lock = self.thread_ready.acquire() > self.code_queue.put(code) > if got_lock: > self.thread_ready.wait() # Wait until processed in timeout interval > else: > print "nolock",source > self.thread_ready.release() > > (and no, using got_lock = self.thread_ready.acquire(blocking=False) > won't fix it). > > Fernando? You remember whether this used to work? Mmh, honestly I don't. This is the problem with not having had a solid test suite ever, we can't even track regressions reliably. That code did change recently: http://projects.scipy.org/ipython/ipython/ticket/210 http://projects.scipy.org/ipython/ipython/ticket/212 and those changes did fix existing deadlocks, but they may have introduced new ones, or simply exposed other latent ones. I'm afraid by now I've shifted my attention enough away from this code that I'd rather just push on the new architecture and leave these in as is, unless someone (marc?) can contribute a good fix. All new code we're writing is nicely factored out and properly tested, and I've finally made a mental shift to the new codebase to the point where I prefer not to work on back fixes unless they are quick and easy. Unfortunately 'quick and easy' does most definitely NOT apply to the threaded logic for all the MT shells :( In summary, if a good samaritan pitches in with a fix I'd be very grateful, but my focus now is 99% on the new code (quick fixes like last night's l?? one aside). Cheers, f From ellisonbg.net at gmail.com Mon Mar 10 15:47:49 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Mon, 10 Mar 2008 13:47:49 -0600 Subject: [IPython-dev] About IPython and bzr In-Reply-To: References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> Message-ID: <6ce0ac130803101247s4355a09dv714d86fd9f24d26e@mail.gmail.com> > > One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough. > > Windows bzr does not support symlinks (even in a half-assed way like > > svn), at least not yet, and I don't know whether such a checkin can be > > reverted. So essentially, if there is even one checkin with symlink, I > > would need to delete the branch and check it in again from local copy. > > Could you investigate these alternatives and let us know how well they work? > > http://bazaar-vcs.org/WindowsSymlinkSupport > https://lists.ubuntu.com/archives/bazaar/2007q1/023801.html > > I'm asking because while we can live for now without symlinks on > trunk, it looks like we'll be using bzr for everything in the future, > and it's likely we may need to use symlinks for certain things. Honest question: I would never have considered even wanting to put symlinks in a repo. Is there a real need for them inside a repo? From vivainio at gmail.com Mon Mar 10 15:48:45 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 10 Mar 2008 21:48:45 +0200 Subject: [IPython-dev] Macros don't work in threaded shells In-Reply-To: References: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com> Message-ID: <46cb515a0803101248x4e7df927v1f1e14a190f85da4@mail.gmail.com> On Mon, Mar 10, 2008 at 9:17 PM, Fernando Perez wrote: > and those changes did fix existing deadlocks, but they may have > introduced new ones, or simply exposed other latent ones. I'm afraid > by now I've shifted my attention enough away from this code that I'd > rather just push on the new architecture and leave these in as is, > unless someone (marc?) can contribute a good fix. All new code we're I'm not totally lost on this - I can just raise a flag when we are about the *really* execute the code, to signal that we are now in thread that should be executing the code. Just wanted to make sure you didn't have sudden "aha" moment. I'll try my fix - it's up to other people (that really stress this threaded shell) to determine how well it works, as I don't really use these MT shells myself. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Mon Mar 10 16:40:59 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 10 Mar 2008 22:40:59 +0200 Subject: [IPython-dev] Macros don't work in threaded shells In-Reply-To: <46cb515a0803101248x4e7df927v1f1e14a190f85da4@mail.gmail.com> References: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com> <46cb515a0803101248x4e7df927v1f1e14a190f85da4@mail.gmail.com> Message-ID: <46cb515a0803101340k3c9c1c43ya4c857103f22b730@mail.gmail.com> On Mon, Mar 10, 2008 at 9:48 PM, Ville M. Vainio wrote: > I'm not totally lost on this - I can just raise a flag when we are > about the *really* execute the code, to signal that we are now in > thread that should be executing the code. I now have a quite straightforward fix for this (that should not break anything at least): http://bazaar.launchpad.net/~ipython/ipython/stable-dev/revision/48 Please try it out, you multithreading parts of the IPython society :-). Try running _ip.runlines("1+1") on your preferred threading solution. However, with these alterations, I think the condition variable and 'got_lock' is an unnecessary complication... but this code is too brittle to play around too much, perhaps. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Mon Mar 10 17:55:23 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 10 Mar 2008 23:55:23 +0200 Subject: [IPython-dev] Macros don't work in threaded shells In-Reply-To: <46cb515a0803101340k3c9c1c43ya4c857103f22b730@mail.gmail.com> References: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com> <46cb515a0803101248x4e7df927v1f1e14a190f85da4@mail.gmail.com> <46cb515a0803101340k3c9c1c43ya4c857103f22b730@mail.gmail.com> Message-ID: <46cb515a0803101455t4bca9df4ycbacd856572956a@mail.gmail.com> On Mon, Mar 10, 2008 at 10:40 PM, Ville M. Vainio wrote: > However, with these alterations, I think the condition variable and > 'got_lock' is an unnecessary complication... but this code is too > brittle to play around too much, perhaps. Though I also figure making it simpler would reduce the danger of deadlocks and other oddities in the future. All we really need is the following structure: - runsource either 1) executes the code directly (if we are in worker thread, i.e. what I just did) or 2) Puts (code, event: threading.Event) tuple in queue, and does even.wait() before it returns Runcode runs code and does event.set() The Queue structure really does all the rest, and I can't see any variables in MTInteractiveShell that need protecting (through using Condition instead of Event) from re-entrant use, esp. since runcode always runs in the same thread. Essentially, the Condition variable now duplicates functionality now already in Queue (in inferior fashion - queue is first come, first served, if there were ipython commands being executed from multiple threads) and makes it harder to understand (I think the condition variable in general is a bit hard to grasp, compared to other threading primitives). Thoughs? Perhaps this is radical enough to be experimented in a separate bzr branch? -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Mon Mar 10 19:01:23 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 10 Mar 2008 16:01:23 -0700 Subject: [IPython-dev] About IPython and bzr In-Reply-To: <6ce0ac130803101247s4355a09dv714d86fd9f24d26e@mail.gmail.com> References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> <6ce0ac130803101247s4355a09dv714d86fd9f24d26e@mail.gmail.com> Message-ID: On Mon, Mar 10, 2008 at 12:47 PM, Brian Granger wrote: > > > One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough. > > > Windows bzr does not support symlinks (even in a half-assed way like > > > svn), at least not yet, and I don't know whether such a checkin can be > > > reverted. So essentially, if there is even one checkin with symlink, I > > > would need to delete the branch and check it in again from local copy. > > > > Could you investigate these alternatives and let us know how well they work? > > > > http://bazaar-vcs.org/WindowsSymlinkSupport > > https://lists.ubuntu.com/archives/bazaar/2007q1/023801.html > > > > I'm asking because while we can live for now without symlinks on > > trunk, it looks like we'll be using bzr for everything in the future, > > and it's likely we may need to use symlinks for certain things. > > Honest question: I would never have considered even wanting to put > symlinks in a repo. Is there a real need for them inside a repo? We had the ChangeLog in so that automatic tools could find it by walking only up the source treee (Emacs does that, I don't know if vi does it also). It's certainly not common enough of a need to warrant losing much sleep over it, and for now we can all just remember NOT to commit symlinks. The fact that the discussion exists for bzr is a good indicator that the need does come up every now and then, but for us it's not something we should worry too much about. If we ever do have a real need for a committed symlink, we'll look into it. Cheers f From fperez.net at gmail.com Mon Mar 10 19:04:12 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 10 Mar 2008 16:04:12 -0700 Subject: [IPython-dev] Macros don't work in threaded shells In-Reply-To: <46cb515a0803101455t4bca9df4ycbacd856572956a@mail.gmail.com> References: <46cb515a0803101153x22d34b91v7f68d0155daddf11@mail.gmail.com> <46cb515a0803101248x4e7df927v1f1e14a190f85da4@mail.gmail.com> <46cb515a0803101340k3c9c1c43ya4c857103f22b730@mail.gmail.com> <46cb515a0803101455t4bca9df4ycbacd856572956a@mail.gmail.com> Message-ID: On Mon, Mar 10, 2008 at 2:55 PM, Ville M. Vainio wrote: > On Mon, Mar 10, 2008 at 10:40 PM, Ville M. Vainio wrote: > > > However, with these alterations, I think the condition variable and > > 'got_lock' is an unnecessary complication... but this code is too > > brittle to play around too much, perhaps. [...] > Thoughs? Perhaps this is radical enough to be experimented in a > separate bzr branch? Thanks for spending some time on it! Because this code is so difficult to debug and test, it's a good idea to put it in a bzr branch. After all, we said we wanted a DVCS to make this type of development model easier, might as well use it :) We might find some time at the Paris sprint to audit this code, and Laurent will be there (he's interested in the WX support), so a branch is perfect to have it waiting til next week. If in the meantime Marc (who contributed the #210 and #212 fixes) has some feedback, that would be great. Cheers, f From bialix at ukr.net Mon Mar 10 19:11:15 2008 From: bialix at ukr.net (Alexander Belchenko) Date: Tue, 11 Mar 2008 01:11:15 +0200 Subject: [IPython-dev] About IPython and bzr In-Reply-To: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> References: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> Message-ID: <47D5C013.7070702@ukr.net> Ville M. Vainio ?????: > My recently-from-scratch installed IPython stable-dev (what used to be > called trunk/) is now very easy to check out from launchpad: > > bzr branch lp:ipython > > One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough. > Windows bzr does not support symlinks (even in a half-assed way like > svn), at least not yet, and I don't know whether such a checkin can be > reverted. `bzr uncommit` allows to uncommit several revisions. Or you can use `bzr branch xxx yyy -r N`, where N -- revision before checkin with symlink. > So essentially, if there is even one checkin with symlink, I > would need to delete the branch and check it in again from local copy. > From tocer.deng at gmail.com Wed Mar 12 05:46:56 2008 From: tocer.deng at gmail.com (tocer) Date: Wed, 12 Mar 2008 17:46:56 +0800 Subject: [IPython-dev] can't %edit python statement inclued non-ascii charset Message-ID: <47D7A690.1040601@gmail.com> hi: I typed a python statement inclued non_ascii charset,such as: In [1] : s = '??' # it's a Chinses string when I wanted to edit the statement again by typing edit command: In [2]: edit In[1] a error occured: UnicodeEncodeError: 'ascii' codec can't encode characters in position 5-6: ordinal not in range(128) how could I make the mistake disapper. btw:my box is ubuntu7.04, python2.5, ipython8.2 thanks. --tocer From vivainio at gmail.com Wed Mar 12 07:25:35 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 12 Mar 2008 13:25:35 +0200 Subject: [IPython-dev] can't %edit python statement inclued non-ascii charset In-Reply-To: <47D7A690.1040601@gmail.com> References: <47D7A690.1040601@gmail.com> Message-ID: <46cb515a0803120425r1b6836cr4855bdbecee367a7@mail.gmail.com> 2008/3/12 tocer : > I typed a python statement inclued non_ascii charset,such as: > > In [1] : s = '??' # it's a Chinses string > > when I wanted to edit the statement again by typing edit command: > > In [2]: edit In[1] > > a error occured: > > UnicodeEncodeError: 'ascii' codec can't encode characters in > position 5-6: ordinal not in range(128) > > how could I make the mistake disapper. The patent answer with unicode errors is, don't do that ;-). The stable IPython neglects unicode on various points, and changing things will cause other parts to break. If you want to provide a solution, create a new branch on bzr about it and try to fix it in Magic.py -> magic_edit. This typically involves encoding/decoding to/from stdin_encoding (grep iplib.py for that to see how it works). However, it's not certain such a fix will be merged into trunk, due to regression risks and the current stability requirements of ipython trunk. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From michael.j.mclay at gmail.com Wed Mar 12 08:22:06 2008 From: michael.j.mclay at gmail.com (Michael McLay) Date: Wed, 12 Mar 2008 08:22:06 -0400 Subject: [IPython-dev] can't %edit python statement inclued non-ascii charset In-Reply-To: <47D7A690.1040601@gmail.com> References: <47D7A690.1040601@gmail.com> Message-ID: <683f83340803120522j5c66379w2a0cd2032ceea665@mail.gmail.com> The error is triggered when the edit command writes the string to a temporary file using the builtin open function. The codecs.open function will write a non-ascii character to a file as the following shows. Python 2.5.1 (r251:54863, May 2 2007, 16:56:35) [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> s = u'\245\200' >>> open('test.uraw', 'w').write(s) Traceback (most recent call last): File "", line 1, in UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128) >>> import codecs >>> codecs.open('test.uraw', 'w',encoding='utf-8').write(s) >>> Andrew Kuchling wrote a very informative tutorial on using Unicode in Python. http://www.amk.ca/python/howto/unicode 2008/3/12 tocer : > hi: > > I typed a python statement inclued non_ascii charset,such as: > > In [1] : s = '??' # it's a Chinses string > > when I wanted to edit the statement again by typing edit command: > > In [2]: edit In[1] > > a error occured: > > UnicodeEncodeError: 'ascii' codec can't encode characters in > position 5-6: ordinal not in range(128) > > how could I make the mistake disapper. > > btw:my box is ubuntu7.04, python2.5, ipython8.2 > > thanks. > > --tocer > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From vivainio at gmail.com Wed Mar 12 08:35:16 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 12 Mar 2008 14:35:16 +0200 Subject: [IPython-dev] can't %edit python statement inclued non-ascii charset In-Reply-To: <683f83340803120522j5c66379w2a0cd2032ceea665@mail.gmail.com> References: <47D7A690.1040601@gmail.com> <683f83340803120522j5c66379w2a0cd2032ceea665@mail.gmail.com> Message-ID: <46cb515a0803120535v1a7dad96h8732d42483b23706@mail.gmail.com> 2008/3/12 Michael McLay : > The error is triggered when the edit command writes the string to a > temporary file using the builtin open function. The codecs.open > function will write a non-ascii character to a file as the following > shows. Ok, then it's probably easy / non-disruptive fix and worth doing. Thanks, I will look into this. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Thu Mar 13 09:55:49 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Thu, 13 Mar 2008 15:55:49 +0200 Subject: [IPython-dev] bzr GUI Message-ID: <46cb515a0803130655j642d57f8g828ace2d7c7aa631@mail.gmail.com> In case someone is looking for a good bzr gui, I recommend QBzr. It's very easy to set up (also on windows), and at least qlog and qannotate (my main use cases) seem to work fine. BTW, if you are running ipython stable-dev branch and sh profile, bzr completer has now been enhanced so that bzr actually gets the commands from "bzr help commands", so you can see what is available in qbzr by "bzr q" -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Thu Mar 13 12:41:53 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 13 Mar 2008 09:41:53 -0700 Subject: [IPython-dev] bzr GUI In-Reply-To: <46cb515a0803130655j642d57f8g828ace2d7c7aa631@mail.gmail.com> References: <46cb515a0803130655j642d57f8g828ace2d7c7aa631@mail.gmail.com> Message-ID: On Thu, Mar 13, 2008 at 6:55 AM, Ville M. Vainio wrote: > In case someone is looking for a good bzr gui, I recommend QBzr. It's > very easy to set up (also on windows), and at least qlog and qannotate > (my main use cases) seem to work fine. Thanks for the tip. Another nice one, that's available out of the box in ubuntu: apt-get install bzr-gtk The actual binary it provides is called 'olive-gtk'. Seems handy so far, especially for viewing the branching history, which is something that a text representation isn't very good at displaying. Cheers, f From michael.j.mclay at gmail.com Thu Mar 13 16:48:41 2008 From: michael.j.mclay at gmail.com (Michael McLay) Date: Thu, 13 Mar 2008 16:48:41 -0400 Subject: [IPython-dev] bzr GUI In-Reply-To: References: <46cb515a0803130655j642d57f8g828ace2d7c7aa631@mail.gmail.com> Message-ID: <683f83340803131348i31a71abbq7c553e4ee676ca7e@mail.gmail.com> I use meld on Ubuntu to look at diffs. It supports bzr. On Thu, Mar 13, 2008 at 12:41 PM, Fernando Perez wrote: > On Thu, Mar 13, 2008 at 6:55 AM, Ville M. Vainio wrote: > > In case someone is looking for a good bzr gui, I recommend QBzr. It's > > very easy to set up (also on windows), and at least qlog and qannotate > > (my main use cases) seem to work fine. > > Thanks for the tip. Another nice one, that's available out of the box > in ubuntu: > > apt-get install bzr-gtk > > The actual binary it provides is called 'olive-gtk'. Seems handy so > far, especially for viewing the branching history, which is something > that a text representation isn't very good at displaying. > > Cheers, > > > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From glenn at tarbox.org Tue Mar 18 14:35:38 2008 From: glenn at tarbox.org (Glenn H Tarbox, PhD) Date: Tue, 18 Mar 2008 11:35:38 -0700 Subject: [IPython-dev] the state if IPython development... Message-ID: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org> I'm having a bit of trouble evaluating various threads of ipython development and am wondering if I'm missing a comms channel. I have various problems which are solveable but don't know which elements of IPython are going to be refactored away in favor of the IPython1 architecture. IPython development has moved, somewhat, to launchpad. I say somewhat both from messages on this list and the "bleeding edge" installation instructions: > bzr co --lightweight lp:ipython > svn co http://ipython.scipy.org/svn/ipython/pyreadline/trunk pyreadline although a "unified" tarball is downloadable from: http://ipython.scipy.org/dist/nightly_svn but, I had trouble with the pyreadline build directly from svn: >>>> copying pyreadline/modes/__init__.py -> build/lib/pyreadline/modes copying pyreadline/modes/notemacs.py -> build/lib/pyreadline/modes package init file 'pyreadline/test/__init__.py' not found (or not a regular file) <<<< I didn't drill down completely but this looks like something inside the package. So, I guess I don't know whether pyreadline within the IPython svn pull is ok, whether one needs to pull both, is there something strange going on when one already has ipython installed?... and don't know whether I should spend any time on this because I can't tell whats OBE. More concerning is the state of Qt4. There is a "critical" bug entered in trac: http://projects.scipy.org/ipython/ipython/ticket/208 Clearly, a QCoreApplication (or QApplication) instance is auto-constructed due to the -q4thread switch... and since calculate.py constructs one also, QT dumps core (why QT decides to core-dump as the error reporting strategy is beyond me... but thats whats going on). Qt4 support is indeed broken... but unnecessarily so. I have Qt4 running with twisted in a naked ipython shell. I simply construct my qt4reactor and use embedded Qt4 code. Can't pull in pylab because it renames packages which collide with PyQt4 (in particular, wrapping QObject in a different module... don't get that yet). and this is beyond the issues associated with QSocketNotifier being invoked from a non-qt thread. I'm sure this is straightforward but, again, I can't determine whether time spent here is worthwhile. Seems to me that IPython1's architecture will make most of these problems "just go away" as its a far more comprehensive architecture. (BTW: the twisted qt reactor can be retrieved from http://noc2.tarbox.org:8080/?p=qtreactor/.git;a=summary ) On a related point, I think we're headed for trouble with wx. I notice that its included in IPython1... but the twisted wxreactor is a bit broken. I think its easy to fix, but the reason its broken is because wx was poorly behaved when the wxreactor was developed (stopped servicing timers, IO notification during modal windows etc) and the twisted folks used threading to get around the difficulty. Nothing wrong with that, other than its ugly (and will cause other threadosity issues) but the code doesn't work properly (fails tests, I support the buildbot) and I wonder whether its worth fixing or whether its easier to get wx to behave (I can drill down into the issues here with anyone who wants but I won't digress further here). Also, I note that there's virtually no traffic on freenode, at least not on #scipy (am I missing a channel?) Seems to me that the IPython1 development activity would be greatly served by greater irc activity. Finally, I support 4 buildbots for twisted. I'm willing to provide these same assets for IPython1 development (each server has dual - 4 core processors = 8 processors / server totaling 32 processors). How to properly use this setup for cluster testing would need to be discussed. -glenn -- Glenn H. Tarbox, PhD glenn at tarbox.org From vivainio at gmail.com Tue Mar 18 16:15:13 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Tue, 18 Mar 2008 22:15:13 +0200 Subject: [IPython-dev] the state if IPython development... In-Reply-To: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org> References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org> Message-ID: <46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com> On Tue, Mar 18, 2008 at 8:35 PM, Glenn H Tarbox, PhD wrote: > IPython development has moved, somewhat, to launchpad. I say somewhat > both from messages on this list and the "bleeding edge" installation > instructions: > > > bzr co --lightweight lp:ipython > > svn co http://ipython.scipy.org/svn/ipython/pyreadline/trunk pyreadline > > although a "unified" tarball is downloadable from: > http://ipython.scipy.org/dist/nightly_svn That tarball only has svn, which has been pretty much discontinued (we will just dump a bzr snapshot there, for curiosity's sake, but you shouldn't count on the svn anymore). > More concerning is the state of Qt4. There is a "critical" bug entered > in trac: http://projects.scipy.org/ipython/ipython/ticket/208 > > Clearly, a QCoreApplication (or QApplication) instance is > auto-constructed due to the -q4thread switch... and since calculate.py > constructs one also, QT dumps core (why QT decides to core-dump as the > error reporting strategy is beyond me... but thats whats going on). > > Qt4 support is indeed broken... but unnecessarily so. I have Qt4 running > with twisted in a naked ipython shell. I simply construct my qt4reactor > and use embedded Qt4 code. Can't pull in pylab because it renames > packages which collide with PyQt4 (in particular, wrapping QObject in a > different module... don't get that yet). and this is beyond the issues > associated with QSocketNotifier being invoked from a non-qt thread. I'm > sure this is straightforward but, again, I can't determine whether time > spent here is worthwhile. Seems to me that IPython1's architecture will > make most of these problems "just go away" as its a far more > comprehensive architecture. IPython1 does not have a magical solution for any of this, it's pretty universal stuff. The problem is still the same - gui mainloop has to be in one thread - but we have that already. Whatever you do in ipython0 regarding Qt4 will be applicable on IPython1 as well, and IPython0 isn't going away anytime soon. Have you considered putting up a bzr branch on launchpad for this stuff? Darren has been most active regarding Qt4, and we could use more hands on this. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From glenn at tarbox.org Tue Mar 18 17:05:06 2008 From: glenn at tarbox.org (Glenn H Tarbox, PhD) Date: Tue, 18 Mar 2008 14:05:06 -0700 Subject: [IPython-dev] the state if IPython development... In-Reply-To: <46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com> References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org> <46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com> Message-ID: <1205874306.17683.158.camel@ghtmyth.hq.tarbox.org> On Tue, 2008-03-18 at 22:15 +0200, Ville M. Vainio wrote: > On Tue, Mar 18, 2008 at 8:35 PM, Glenn H Tarbox, PhD wrote: > That tarball only has svn, which has been pretty much discontinued (we > will just dump a bzr snapshot there, for curiosity's sake, but you > shouldn't count on the svn anymore). we should probably indicate this somewhere. There is very little information being disseminated WRT where one should go for the bleeding edge (which is the only place I feel comfortable :-) > > > More concerning is the state of Qt4. There is a "critical" bug entered > > in trac: http://projects.scipy.org/ipython/ipython/ticket/208 > > > > Clearly, a QCoreApplication (or QApplication) instance is > > auto-constructed due to the -q4thread switch... and since calculate.py > > constructs one also, QT dumps core (why QT decides to core-dump as the > > error reporting strategy is beyond me... but thats whats going on). > > > > Qt4 support is indeed broken... but unnecessarily so. I have Qt4 running > > with twisted in a naked ipython shell. I simply construct my qt4reactor > > and use embedded Qt4 code. Can't pull in pylab because it renames > > packages which collide with PyQt4 (in particular, wrapping QObject in a > > different module... don't get that yet). and this is beyond the issues > > associated with QSocketNotifier being invoked from a non-qt thread. I'm > > sure this is straightforward but, again, I can't determine whether time > > spent here is worthwhile. Seems to me that IPython1's architecture will > > make most of these problems "just go away" as its a far more > > comprehensive architecture. > > IPython1 does not have a magical solution for any of this, it's pretty > universal stuff. The problem is still the same - gui mainloop has to > be in one thread - but we have that already. Whatever you do in > ipython0 regarding Qt4 will be applicable on IPython1 as well, and > IPython0 isn't going away anytime soon. Actually, this is fairly straightforward with Qt and, I believe, the other gui mainloops. Since you're embracing twisted, I recommend adding stdin and stdout to the event notification hooks and you're mostly done. This is what I'm (kinda) doing... where things fall down a bit is when IPython grabs the loop during scrolling (for example when reading the docs which spew with a '??'.) Other than that, it just kinda works. Of course, I believe this is part of the new / better / different IPython1 design. Since IPython0 is gonna hang around we need to decide whether to work deep inside to address this issue. I previously posted a workaround to nail twisted to IPython0 which exploited the multithreaded IShell code. So, on this issue, we need a strategy. I certainly appreciate continued support for IPython0, but given the limited resources I wonder how much effort should be put toward fixing the various GUI loops. As it stands, Qt4 is currently broken with IPython0 running pylab. The decision to expend effort needs to be based on the development plan / schedule. This is something I have no insight into as there is no chatter I have access to. and I'll say again, you're in the soup with wx. It might work for a while, but its broken and needs work. I looked at it with the twisted team but decided to just bag it and go Qt. In that effort, it became clear how to fix wx which is when I became aware of wx's previous issues which may or may not still exist. The use of threads to get around wx blocking is, IMHO, a bad idea... much better to fix wx... and if that can't be done, it should be bagged or someone needs to take on that project, certainly beyond my time availability. > > Have you considered putting up a bzr branch on launchpad for this > stuff? Darren has been most active regarding Qt4, and we could use > more hands on this. Love to... but this further emphasizes my point. I have no idea whats going on WRT development of the various flavors of IPython. I have a feeling I'm not the only one without the necessary visibility. > -- Glenn H. Tarbox, PhD glenn at tarbox.org From fperez.net at gmail.com Tue Mar 18 19:26:00 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 18 Mar 2008 16:26:00 -0700 Subject: [IPython-dev] the state if IPython development... In-Reply-To: <1205874306.17683.158.camel@ghtmyth.hq.tarbox.org> References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org> <46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com> <1205874306.17683.158.camel@ghtmyth.hq.tarbox.org> Message-ID: Hi Glenn, On Tue, Mar 18, 2008 at 2:05 PM, Glenn H Tarbox, PhD wrote: > On Tue, 2008-03-18 at 22:15 +0200, Ville M. Vainio wrote: > > On Tue, Mar 18, 2008 at 8:35 PM, Glenn H Tarbox, PhD > wrote: > > > > That tarball only has svn, which has been pretty much discontinued (we > > will just dump a bzr snapshot there, for curiosity's sake, but you > > shouldn't count on the svn anymore). > > we should probably indicate this somewhere. There is very little > information being disseminated WRT where one should go for the bleeding > edge (which is the only place I feel comfortable :-) I take full responsibility for poor communications recently. In my defense, I've been traveling a lot and coding, and most of this work will directly or indirectly benefit ipython: - we had a sprint on sage/scipy at Enthought's headquarters in Austin in early March where several ipython devs also worked a lot, with great progress made by Brian and Min on a number of issues. We were also joined by Ondrej Certik (of sympy fame) and Stefan van der Walt (from numpy/scipy). - I returned to Berkeley where I've been working with Stefan on some prototype algorithms for an NSF grant proposal which, if funded, would benefit ipython. It's on mathematics and neuroscience, but the core implementation would all be in scipy/ipython. - We are right now in Paris holding a development meeting related to scipy and neuroimaging in Python; this is part of my 'real job' but it also involves ipython work and resources (I ran a demo this morning of ipython1 and some local labs are already testing things out for their work). - This weekend we'll hold a mini-sprint at Gael Varoquaux's house in Paris (from Mayavi2), to push forward on a number of key ipython1/ipython-trunk integration issues. So in summary, I've just been so swamped with this work that I've done a poor job of communicating it to the rest of the team, except for all the face-to-face conversations we've had. Perhaps this will convince me to get a blog: for the first time in my life I see the point of those things. While I never thought anyone would have a reason to be interested in my personal ramblings, it may be a good way to occasionally post things that aren't exactly pure mailing list announcements. By aggregating the posts at a place like http://planet.scipy.org, interested readers can follow the appropriately tagged posts. I'm afraid after this mini-update I'll disappear again; I need some sleep and we have a long day ahead tomorrow. I hope you'll forgive the incompleteness of this post, but I realize that we need to do a better job of community building. If you think that a blog would be the right resource (I know Ville already has one) to help on this front, let me know as I'd appreciate the feedback. Cheers, f > > > > > > > More concerning is the state of Qt4. There is a "critical" bug > entered > > > in trac: http://projects.scipy.org/ipython/ipython/ticket/208 > > > > > > Clearly, a QCoreApplication (or QApplication) instance is > > > auto-constructed due to the -q4thread switch... and since > calculate.py > > > constructs one also, QT dumps core (why QT decides to core-dump as > the > > > error reporting strategy is beyond me... but thats whats going on). > > > > > > Qt4 support is indeed broken... but unnecessarily so. I have Qt4 > running > > > with twisted in a naked ipython shell. I simply construct my > qt4reactor > > > and use embedded Qt4 code. Can't pull in pylab because it renames > > > packages which collide with PyQt4 (in particular, wrapping QObject > in a > > > different module... don't get that yet). and this is beyond the > issues > > > associated with QSocketNotifier being invoked from a non-qt thread. > I'm > > > sure this is straightforward but, again, I can't determine whether > time > > > spent here is worthwhile. Seems to me that IPython1's architecture > will > > > make most of these problems "just go away" as its a far more > > > comprehensive architecture. > > > > IPython1 does not have a magical solution for any of this, it's pretty > > universal stuff. The problem is still the same - gui mainloop has to > > be in one thread - but we have that already. Whatever you do in > > ipython0 regarding Qt4 will be applicable on IPython1 as well, and > > IPython0 isn't going away anytime soon. > > Actually, this is fairly straightforward with Qt and, I believe, the > other gui mainloops. Since you're embracing twisted, I recommend adding > stdin and stdout to the event notification hooks and you're mostly done. > This is what I'm (kinda) doing... where things fall down a bit is when > IPython grabs the loop during scrolling (for example when reading the > docs which spew with a '??'.) Other than that, it just kinda works. > > Of course, I believe this is part of the new / better / different > IPython1 design. > > Since IPython0 is gonna hang around we need to decide whether to work > deep inside to address this issue. I previously posted a workaround to > nail twisted to IPython0 which exploited the multithreaded IShell code. > > So, on this issue, we need a strategy. I certainly appreciate continued > support for IPython0, but given the limited resources I wonder how much > effort should be put toward fixing the various GUI loops. > > As it stands, Qt4 is currently broken with IPython0 running pylab. The > decision to expend effort needs to be based on the development plan / > schedule. This is something I have no insight into as there is no > chatter I have access to. > > and I'll say again, you're in the soup with wx. It might work for a > while, but its broken and needs work. I looked at it with the twisted > team but decided to just bag it and go Qt. In that effort, it became > clear how to fix wx which is when I became aware of wx's previous issues > which may or may not still exist. The use of threads to get around wx > blocking is, IMHO, a bad idea... much better to fix wx... and if that > can't be done, it should be bagged or someone needs to take on that > project, certainly beyond my time availability. > > > > > > Have you considered putting up a bzr branch on launchpad for this > > stuff? Darren has been most active regarding Qt4, and we could use > > more hands on this. > > Love to... but this further emphasizes my point. I have no idea whats > going on WRT development of the various flavors of IPython. I have a > feeling I'm not the only one without the necessary visibility. > > > > -- > > Glenn H. Tarbox, PhD > glenn at tarbox.org > > > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From vivainio at gmail.com Wed Mar 19 02:37:20 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 19 Mar 2008 08:37:20 +0200 Subject: [IPython-dev] the state if IPython development... In-Reply-To: <1205873455.17683.153.camel@ghtmyth.hq.tarbox.org> References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org> <46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com> <1205873455.17683.153.camel@ghtmyth.hq.tarbox.org> Message-ID: <46cb515a0803182337i7efc23b7w1bf14f7770586163@mail.gmail.com> On Tue, Mar 18, 2008 at 10:50 PM, Glenn H Tarbox, PhD wrote: > On Tue, 2008-03-18 at 22:15 +0200, Ville M. Vainio wrote: > > On Tue, Mar 18, 2008 at 8:35 PM, Glenn H Tarbox, PhD wrote: > > > > That tarball only has svn, which has been pretty much discontinued (we > > will just dump a bzr snapshot there, for curiosity's sake, but you > > shouldn't count on the svn anymore). > > we should probably indicate this somewhere. There is very little > information being disseminated WRT where one should go for the bleeding > edge (which is the only place I feel comfortable :-) Thea deal is that there are 2 bleeding edges - the bzr of stable-dev and bzr of IPython1. IPython1 is not the bleeding edge of IPython0, even if both projects have "IPython" in it ;-). Think of this as baz and bzr, standard readline and pyreadline, glade and gazpacho... > Actually, this is fairly straightforward with Qt and, I believe, the > other gui mainloops. Since you're embracing twisted, I recommend adding > stdin and stdout to the event notification hooks and you're mostly done. > This is what I'm (kinda) doing... where things fall down a bit is when > IPython grabs the loop during scrolling (for example when reading the > docs which spew with a '??'.) Other than that, it just kinda works. Sound great. If you have something in place, please share the code! Do new applications you %run reuse the reactor? > Of course, I believe this is part of the new / better / different > IPython1 design. Yes, but we are not opposed to this in IPython0 either. The ground rule is that as opposed to breaking stuff, we can add stuff. > Since IPython0 is gonna hang around we need to decide whether to work > deep inside to address this issue. I previously posted a workaround to Depends how deep inside you need to go. If it needs a lot of effort (10+ hours), then the change itself may be too big for IPython0... One factor is also how fast you want the thing to be in the hands of the end users. > So, on this issue, we need a strategy. I certainly appreciate continued > support for IPython0, but given the limited resources I wonder how much > effort should be put toward fixing the various GUI loops. This is something I've been pondering as well, I've been thinking of a simpler approach to handle these. One idea has been introducing a magic command, say, %goqt4 that would register IPython with an already existing Qt4 application - as opposed to the current approach of constructing a QApplication of our own. Alternatively, we could monkeypatch QApplication so that when it is created, we do our own stuff for it and return (which is similar to how we handle mainloops now). I haven't really looked deep into this since I tend not to use these much. > As it stands, Qt4 is currently broken with IPython0 running pylab. The > decision to expend effort needs to be based on the development plan / > schedule. This is something I have no insight into as there is no > chatter I have access to. You are talking about pylab_import_all option where it gets all the names? This can be fixed easily. Just give the details. > > Have you considered putting up a bzr branch on launchpad for this > > stuff? Darren has been most active regarding Qt4, and we could use > > more hands on this. > > Love to... but this further emphasizes my point. I have no idea whats > going on WRT development of the various flavors of IPython. I have a > feeling I'm not the only one without the necessary visibility. Just branch stable-dev branch and play around, if you need something usable soon. IPython1 is still on "planning" stage regarding how to run GUI mainloops in IPython core, as far as I understand. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ondrej at certik.cz Wed Mar 19 15:53:59 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Wed, 19 Mar 2008 20:53:59 +0100 Subject: [IPython-dev] the state if IPython development... In-Reply-To: References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org> <46cb515a0803181315k3f25bff9p1687fb71c27c078e@mail.gmail.com> <1205874306.17683.158.camel@ghtmyth.hq.tarbox.org> Message-ID: <85b5c3130803191253j22f276c4w635aedef55695821@mail.gmail.com> On Wed, Mar 19, 2008 at 12:26 AM, Fernando Perez wrote: > Hi Glenn, > > > On Tue, Mar 18, 2008 at 2:05 PM, Glenn H Tarbox, PhD wrote: > > On Tue, 2008-03-18 at 22:15 +0200, Ville M. Vainio wrote: > > > On Tue, Mar 18, 2008 at 8:35 PM, Glenn H Tarbox, PhD > > wrote: > > > > > > > That tarball only has svn, which has been pretty much discontinued (we > > > will just dump a bzr snapshot there, for curiosity's sake, but you > > > shouldn't count on the svn anymore). > > > > we should probably indicate this somewhere. There is very little > > information being disseminated WRT where one should go for the bleeding > > edge (which is the only place I feel comfortable :-) > > I take full responsibility for poor communications recently. In my > defense, I've been traveling a lot and coding, and most of this work > will directly or indirectly benefit ipython: > > - we had a sprint on sage/scipy at Enthought's headquarters in Austin > in early March where several ipython devs also worked a lot, with > great progress made by Brian and Min on a number of issues. We were > also joined by Ondrej Certik (of sympy fame) and Stefan van der Walt > (from numpy/scipy). > > - I returned to Berkeley where I've been working with Stefan on some > prototype algorithms for an NSF grant proposal which, if funded, would > benefit ipython. It's on mathematics and neuroscience, but the core > implementation would all be in scipy/ipython. > > - We are right now in Paris holding a development meeting related to > scipy and neuroimaging in Python; this is part of my 'real job' but it > also involves ipython work and resources (I ran a demo this morning of > ipython1 and some local labs are already testing things out for their > work). > > - This weekend we'll hold a mini-sprint at Gael Varoquaux's house in > Paris (from Mayavi2), to push forward on a number of key > ipython1/ipython-trunk integration issues. > > So in summary, I've just been so swamped with this work that I've done > a poor job of communicating it to the rest of the team, except for all > the face-to-face conversations we've had. > > Perhaps this will convince me to get a blog: for the first time in my > life I see the point of those things. While I never thought anyone > would have a reason to be interested in my personal ramblings, it may > be a good way to occasionally post things that aren't exactly pure > mailing list announcements. By aggregating the posts at a place like > http://planet.scipy.org, interested readers can follow the > appropriately tagged posts. > > I'm afraid after this mini-update I'll disappear again; I need some > sleep and we have a long day ahead tomorrow. > > I hope you'll forgive the incompleteness of this post, but I realize > that we need to do a better job of community building. If you think > that a blog would be the right resource (I know Ville already has one) > to help on this front, let me know as I'd appreciate the feedback. Yes, I think having a blog is an extremely good way of telling the world about your intentions and also about what has been done. Ondrej From vivainio at gmail.com Wed Mar 19 17:39:28 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 19 Mar 2008 23:39:28 +0200 Subject: [IPython-dev] Twisted reactor + IPython0 Message-ID: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> This is related to the recent thread about reactors, GUI support etc. Would it make sense to provide direct twisted reactor support for ipython0? There is a recipe for this: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410670 If this works as expected, can't we just implement all the GUI support widgets in terms of this baby, using the support bundled with twisted? A couple of you have gathered extensive experience with Twisted, what do you think? -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From glenn at tarbox.org Wed Mar 19 18:35:16 2008 From: glenn at tarbox.org (Glenn H Tarbox, PhD) Date: Wed, 19 Mar 2008 15:35:16 -0700 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> Message-ID: <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> This is where I started originally. There's a better fix which I posted a while ago. The below uses polling which is unnecessary. By modifying shell, you can turn the whole thing into a callback. A better approach is to use my qtreactor (it sucks in qt but) because you can instantiate twisted (and qt) and keep the event loop for stdin/stdout and have reactor behavior as well. the key with qtreactor is I made a new call = reactor.runReturn() which nails up the reactor to the Qt application, sets up the timers etc, but returns the thread to you... so as long as I/O in IPython doesn't block (which it only does when scrolling more than a page) -glenn On Wed, 2008-03-19 at 23:39 +0200, Ville M. Vainio wrote: > This is related to the recent thread about reactors, GUI support etc. > > Would it make sense to provide direct twisted reactor support for ipython0? > > There is a recipe for this: > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410670 > > If this works as expected, can't we just implement all the GUI support > widgets in terms of this baby, using the support bundled with twisted? > > A couple of you have gathered extensive experience with Twisted, what > do you think? > -- Glenn H. Tarbox, PhD 206-494-0819 glenn at tarbox.org From glenn at tarbox.org Wed Mar 19 18:51:34 2008 From: glenn at tarbox.org (Glenn H Tarbox, PhD) Date: Wed, 19 Mar 2008 15:51:34 -0700 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> Message-ID: <1205967094.26877.111.camel@ghtmyth.hq.tarbox.org> to wit: import sys from PyQt4 import QtGui import qt4reactor """ this nails up the reactor and makes app available at the module level """ app = QtGui.QApplication(sys.argv) qt4reactor.install() # alternative for those who don't need a gui # from twisted.application import reactors # reactors.installReactor('qt4') from twisted.internet import reactor reactor.runReturn() # now you have the thread back, a live Qt application object, and # twisted... and you've got the IPython0 prompt back... then try def callMe(): print "he called me" reactor.callLater(5,0,callMe) you'll have the prompt back and 5 seconds later it'll print On Wed, 2008-03-19 at 15:35 -0700, Glenn H Tarbox, PhD wrote: > This is where I started originally. There's a better fix which I posted > a while ago. The below uses polling which is unnecessary. By modifying > shell, you can turn the whole thing into a callback. > > A better approach is to use my qtreactor (it sucks in qt but) because > you can instantiate twisted (and qt) and keep the event loop for > stdin/stdout and have reactor behavior as well. > > the key with qtreactor is I made a new call = reactor.runReturn() which > nails up the reactor to the Qt application, sets up the timers etc, but > returns the thread to you... so as long as I/O in IPython doesn't block > (which it only does when scrolling more than a page) > > -glenn > > > On Wed, 2008-03-19 at 23:39 +0200, Ville M. Vainio wrote: > > This is related to the recent thread about reactors, GUI support etc. > > > > Would it make sense to provide direct twisted reactor support for ipython0? > > > > There is a recipe for this: > > > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410670 > > > > If this works as expected, can't we just implement all the GUI support > > widgets in terms of this baby, using the support bundled with twisted? > > > > A couple of you have gathered extensive experience with Twisted, what > > do you think? > > -- Glenn H. Tarbox, PhD 206-494-0819 glenn at tarbox.org From barrywark at gmail.com Wed Mar 19 19:51:19 2008 From: barrywark at gmail.com (Barry Wark) Date: Wed, 19 Mar 2008 16:51:19 -0700 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> Message-ID: On Wed, Mar 19, 2008 at 2:39 PM, Ville M. Vainio wrote: > This is related to the recent thread about reactors, GUI support etc. > > Would it make sense to provide direct twisted reactor support for ipython0? > > There is a recipe for this: > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410670 > > If this works as expected, can't we just implement all the GUI support > widgets in terms of this baby, using the support bundled with twisted? > > A couple of you have gathered extensive experience with Twisted, what > do you think? I'm definitely not an IPython1 or Twisted expert (more of a newbie) and I know next to nothing about the IPython0 codebase, so take my comments with a grain of salt. My understanding is that this is how GUI support can be added to IPython1. ipython1.kernel.engineservice exposes an IEngine interface as a Twisted service. The IEngine interface provides the functionality of (and the implementation delegates to) the ipython core. Thus GUI interfaces can interact with the ipython1 core via Twisted. This is the approach I've used in the ipython1-cocoa branch to provide a Cocoa frontend for the ipython1 core (i.e. via ipython1.kernel.engineservice.IEngineInteractive). Barry From ellisonbg.net at gmail.com Wed Mar 19 23:10:16 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 19 Mar 2008 21:10:16 -0600 Subject: [IPython-dev] the state if IPython development... In-Reply-To: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org> References: <1205865338.17683.116.camel@ghtmyth.hq.tarbox.org> Message-ID: <6ce0ac130803192010j62fc5d7as932566b97140b8e2@mail.gmail.com> > I'm having a bit of trouble evaluating various threads of ipython > development and am wondering if I'm missing a comms channel. I have > various problems which are solveable but don't know which elements of > IPython are going to be refactored away in favor of the IPython1 > architecture. Sorry about this, it hasn't helped that I have been traveling a lot recently and sick as well. > More concerning is the state of Qt4. There is a "critical" bug entered > in trac: http://projects.scipy.org/ipython/ipython/ticket/208 The Qt4 bug in IPython0 should be fixed. Period. We just need someone to tackle it. There are just too many people who use it to stop fixing bugs like this. Our overall plan is to continue supporting IPython0 until IPython1 is a full replacement. > Clearly, a QCoreApplication (or QApplication) instance is > auto-constructed due to the -q4thread switch... and since calculate.py > constructs one also, QT dumps core (why QT decides to core-dump as the > error reporting strategy is beyond me... but thats whats going on). > > Qt4 support is indeed broken... but unnecessarily so. I have Qt4 running > with twisted in a naked ipython shell. I simply construct my qt4reactor > and use embedded Qt4 code. Can't pull in pylab because it renames > packages which collide with PyQt4 (in particular, wrapping QObject in a > different module... don't get that yet). and this is beyond the issues > associated with QSocketNotifier being invoked from a non-qt thread. I'm > sure this is straightforward but, again, I can't determine whether time > spent here is worthwhile. Seems to me that IPython1's architecture will > make most of these problems "just go away" as its a far more > comprehensive architecture. > > (BTW: the twisted qt reactor can be retrieved from > http://noc2.tarbox.org:8080/?p=qtreactor/.git;a=summary ) > > On a related point, I think we're headed for trouble with wx. I notice > that its included in IPython1... but the twisted wxreactor is a bit > broken. I think its easy to fix, but the reason its broken is because > wx was poorly behaved when the wxreactor was developed (stopped > servicing timers, IO notification during modal windows etc) and the > twisted folks used threading to get around the difficulty. Nothing > wrong with that, other than its ugly (and will cause other threadosity > issues) but the code doesn't work properly (fails tests, I support the > buildbot) and I wonder whether its worth fixing or whether its easier to > get wx to behave (I can drill down into the issues here with anyone who > wants but I won't digress further here). What is included with IPython1 (wx is not?). We do need to have some discussions about how we will handle GUI integration within IPython1. This is a much bigger discussion though - probably need to start a new thread for this. > Finally, I support 4 buildbots for twisted. I'm willing to provide > these same assets for IPython1 development (each server has dual - 4 > core processors = 8 processors / server totaling 32 processors). How to > properly use this setup for cluster testing would need to be discussed. For now the IPython1 test suite does not need multiple processors (we are using twisted's trial). But, we surely could use an IPython1 buildbot. Eventually we will definitely probably have multiprocess tests though. > -glenn > > -- > Glenn H. Tarbox, PhD > glenn at tarbox.org > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Wed Mar 19 23:21:48 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 19 Mar 2008 21:21:48 -0600 Subject: [IPython-dev] A few notes on threading+GUI event loops in Ipython1 Message-ID: <6ce0ac130803192021x235aa192s727c6f8f34f1d961@mail.gmail.com> Hi, With all the recent activity about GUi event loops + threading, I figured I would jot down a few notes about the current state of GUI event loop integration within IPython1. 1) The IPython1 engine/core can be run in two different modes: with the twisted event loop (typically when it runs in another process) or without it. Thus, we will need to have support for GUI event loop integration that works for both of these cases. 2) If the twisted event loop is running, one option is to use one of the twisted reactors like the wxreactor. But, we will need to carefully study if these reactors really work (there seems to be some evidence that there are problems). 3) The issue of threads in the IPython1 engine could get _very_ complicated (much more so even that the treading in ipython0). Part of the problem is that there is simply many more possibilities for how the ipython1 engine can be run. It is probably too early to tackle these things, but we are getting close to needing to think about these things. Brian From ellisonbg.net at gmail.com Wed Mar 19 23:21:48 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 19 Mar 2008 21:21:48 -0600 Subject: [IPython-dev] A few notes on threading+GUI event loops in Ipython1 Message-ID: <6ce0ac130803192021x235aa192s727c6f8f34f1d961@mail.gmail.com> Hi, With all the recent activity about GUi event loops + threading, I figured I would jot down a few notes about the current state of GUI event loop integration within IPython1. 1) The IPython1 engine/core can be run in two different modes: with the twisted event loop (typically when it runs in another process) or without it. Thus, we will need to have support for GUI event loop integration that works for both of these cases. 2) If the twisted event loop is running, one option is to use one of the twisted reactors like the wxreactor. But, we will need to carefully study if these reactors really work (there seems to be some evidence that there are problems). 3) The issue of threads in the IPython1 engine could get _very_ complicated (much more so even that the treading in ipython0). Part of the problem is that there is simply many more possibilities for how the ipython1 engine can be run. It is probably too early to tackle these things, but we are getting close to needing to think about these things. Brian From vivainio at gmail.com Thu Mar 20 01:33:00 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Thu, 20 Mar 2008 07:33:00 +0200 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> Message-ID: <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> On Thu, Mar 20, 2008 at 12:35 AM, Glenn H Tarbox, PhD wrote: > This is where I started originally. There's a better fix which I posted > a while ago. The below uses polling which is unnecessary. By modifying > shell, you can turn the whole thing into a callback. The ASPN recipe works like the current GUI backends, so it seems to "obviously work". > > A better approach is to use my qtreactor (it sucks in qt but) because > you can instantiate twisted (and qt) and keep the event loop for > stdin/stdout and have reactor behavior as well. Would that work with other backends as well? Does it work with windows + pyreadline? > the key with qtreactor is I made a new call = reactor.runReturn() which > nails up the reactor to the Qt application, sets up the timers etc, but > returns the thread to you... so as long as I/O in IPython doesn't block > (which it only does when scrolling more than a page) That's a pretty fine solution, if it can be made to work. Can you produce a code snippet that does this for IPython0? -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From glenn at tarbox.org Thu Mar 20 01:41:18 2008 From: glenn at tarbox.org (Glenn H Tarbox, PhD) Date: Wed, 19 Mar 2008 22:41:18 -0700 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> Message-ID: <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> On Thu, 2008-03-20 at 07:33 +0200, Ville M. Vainio wrote: > On Thu, Mar 20, 2008 at 12:35 AM, Glenn H Tarbox, PhD wrote: > > > This is where I started originally. There's a better fix which I posted > > a while ago. The below uses polling which is unnecessary. By modifying > > shell, you can turn the whole thing into a callback. > > The ASPN recipe works like the current GUI backends, so it seems to > "obviously work". it works, just spins cycles unnecessarily. If you look through this list you'll see the minor changes I made to Shell (posted on Feb 9). My code generates a callback when somethings needs to be serviced as opposed to having a high-frequency polling loop try and determine the same thing > > > > > A better approach is to use my qtreactor (it sucks in qt but) because > > you can instantiate twisted (and qt) and keep the event loop for > > stdin/stdout and have reactor behavior as well. > > Would that work with other backends as well? Does it work with windows > + pyreadline? yup > > > the key with qtreactor is I made a new call = reactor.runReturn() which > > nails up the reactor to the Qt application, sets up the timers etc, but > > returns the thread to you... so as long as I/O in IPython doesn't block > > (which it only does when scrolling more than a page) > > That's a pretty fine solution, if it can be made to work. > > Can you produce a code snippet that does this for IPython0? was already posted before you wrote this. > -- Glenn H. Tarbox, PhD 206-494-0819 glenn at tarbox.org From vivainio at gmail.com Thu Mar 20 02:20:15 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Thu, 20 Mar 2008 08:20:15 +0200 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> Message-ID: <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> On Thu, Mar 20, 2008 at 7:41 AM, Glenn H Tarbox, PhD wrote: > On Thu, 2008-03-20 at 07:33 +0200, Ville M. Vainio wrote: > > On Thu, Mar 20, 2008 at 12:35 AM, Glenn H Tarbox, PhD wrote: > > > > > This is where I started originally. There's a better fix which I posted > > > a while ago. The below uses polling which is unnecessary. By modifying > > > shell, you can turn the whole thing into a callback. > > > > The ASPN recipe works like the current GUI backends, so it seems to > > "obviously work". > > it works, just spins cycles unnecessarily. If you look through this > list you'll see the minor changes I made to Shell (posted on Feb 9). My > code generates a callback when somethings needs to be serviced as > opposed to having a high-frequency polling loop try and determine the > same thing Goodie. Frankly, I probably missed the original discussion because I thought of it was mostly related to IPython1 (though obviously I can't remember why I missed it) ;-) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From laurent.dufrechou at gmail.com Thu Mar 20 04:13:56 2008 From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=) Date: Thu, 20 Mar 2008 09:13:56 +0100 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> Message-ID: <47e21ccf.0aaa660a.1579.1217@mx.google.com> Hi all, Just to drop some more need on threading. I've written a little piece of code to create a WX Ipython shell. I've used for this a thread with an ipython0 instance and communicate with the gui via polling (the first version) and now I'm starting to use callback. One big need I have and that cannot be resolved easily is to ba able to have some sort of Ipython instance with GUI loop support + twisted support. The idea: Any WXAPP/QTApp <-- XML RPC --> The twisted loop + Any GUI(QT/WX...) loop + Ipython server. With XML RPC you can just make any action inside Ipython instance. I want this to be able to make a WX(and QT) console able to launch any QT/Wx windows and thus independent to the Console App. With this you can have a really really good dev env for graphical app. Moreover designing such a server will be raally good for people who want an ipython console, because with this all the wx/QT/... console code will share the same ipython object that will save lot of work. Glenn, perhaps my idea is stupid, but do you thing we will have a lot of performance related problem with this, do you see any issue as you've worked a little on something like this?? Perhaps it is a too big thing for a simple Ipython console... Laurent -----Message d'origine----- De?: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De la part de Ville M. Vainio Envoy??: jeudi 20 mars 2008 07:20 ??: Glenn H Tarbox, PhD Cc?: ipython-dev Mailing list Objet?: Re: [IPython-dev] Twisted reactor + IPython0 On Thu, Mar 20, 2008 at 7:41 AM, Glenn H Tarbox, PhD wrote: > On Thu, 2008-03-20 at 07:33 +0200, Ville M. Vainio wrote: > > On Thu, Mar 20, 2008 at 12:35 AM, Glenn H Tarbox, PhD wrote: > > > > > This is where I started originally. There's a better fix which I posted > > > a while ago. The below uses polling which is unnecessary. By modifying > > > shell, you can turn the whole thing into a callback. > > > > The ASPN recipe works like the current GUI backends, so it seems to > > "obviously work". > > it works, just spins cycles unnecessarily. If you look through this > list you'll see the minor changes I made to Shell (posted on Feb 9). My > code generates a callback when somethings needs to be serviced as > opposed to having a high-frequency polling loop try and determine the > same thing Goodie. Frankly, I probably missed the original discussion because I thought of it was mostly related to IPython1 (though obviously I can't remember why I missed it) ;-) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' _______________________________________________ IPython-dev mailing list IPython-dev at scipy.org http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev No virus found in this incoming message. Checked by AVG. Version: 7.5.519 / Virus Database: 269.21.7/1334 - Release Date: 18/03/2008 20:52 No virus found in this outgoing message. Checked by AVG. Version: 7.5.519 / Virus Database: 269.21.7/1334 - Release Date: 18/03/2008 20:52 From vivainio at gmail.com Thu Mar 20 04:33:48 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Thu, 20 Mar 2008 10:33:48 +0200 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <47e21ccf.0aaa660a.1579.1217@mx.google.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> Message-ID: <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> On Thu, Mar 20, 2008 at 10:13 AM, Laurent Dufr?chou wrote: > With this you can have a really really good dev env for graphical app. > Moreover designing such a server will be raally good for people who want an > ipython console, because with this all the wx/QT/... console code will share > the same ipython object that will save lot of work. > > Glenn, perhaps my idea is stupid, but do you thing we will have a lot of > performance related problem with this, do you see any issue as you've worked > a little on something like this?? > Perhaps it is a too big thing for a simple Ipython console... It really isn't, once we get the twisted reactor to be the "core" of IPython0. There is lots of fun to be had with this. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Thu Mar 20 06:07:01 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 20 Mar 2008 03:07:01 -0700 Subject: [IPython-dev] Discussion on Bruce Eckel's blog Message-ID: Hi all, this may interest several of us here, since ipython is mentioned more than once. By pycon'09 we'll have much more than 'claims' to present :) http://www.artima.com/forums/flat.jsp?forum=106&thread=227073&start=0 Cheers, f From glenn at tarbox.org Thu Mar 20 10:15:14 2008 From: glenn at tarbox.org (Glenn H Tarbox, PhD) Date: Thu, 20 Mar 2008 07:15:14 -0700 Subject: [IPython-dev] Discussion on Bruce Eckel's blog In-Reply-To: References: Message-ID: <1206022514.3359.71.camel@ghtmyth.hq.tarbox.org> Perhaps its early and maybe I should wait a couplea hours before responding to this... My understanding is that Bruce Eckel is a serious guy and someone due respect... But the post is silly. I really don't know where to start and I think he should just delete it and we should forget it exists. Is he trying to discuss the issues of parallelism, threading, co-routine abstractions, inter process-communication? In the same brief blurb he discusses and mixes all four. Makes mention of the GIL and how java multithreading through co-routines (stackless???) are... what? I got confused even parsing the words. none of these issues are related at the level of the post. Stack frames, threads, processes, IPC and netowrking are, clearly, related but wildly different subjects... not to mention that none of these have anything to do with solving parallel computing problems other than we might want to use multiple computers and electricity... Is the point that some of the packages mentioned make serializing objects easier? And IPython1? Sure, there's a notion of a redesign to embrace actual parallel computation issues, and even a couplea examples of how one might use asynchronous models to implement map-reduce (or scatter-gather or whatever) But the IPython1 effort is mostly about embracing the asynchronous model (read Twisted) with a sugary sweet layer of IPython to make it a bit less painful. Unless I missed a bunch of documentation somewhere (certainly possible given the state of the documentation), this hardly rises to: "a powerful architecture for parallel computing," So, please, lets not get into a debate over this article. Its trivial and muddled. and lets certainly not believe our own line of BS if someone did say something like this to Bruce. IPython1 has a notion that there should be an architecture and even some early examples... but there's no new model, no power other than doing stuff correctly with modern tools. Hell, I'm not even sure Bruce got the relevant points about Numpy right... maybe he spent a few too many hours in the lobby bar and needed to spew a pre-defined number of words to justify the travel expense. -glenn -- Glenn H. Tarbox, PhD 206-494-0819 glenn at tarbox.org On Thu, 2008-03-20 at 03:07 -0700, Fernando Perez wrote: > Hi all, > > this may interest several of us here, since ipython is mentioned more > than once. By pycon'09 we'll have much more than 'claims' to present > :) > > http://www.artima.com/forums/flat.jsp?forum=106&thread=227073&start=0 > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org http://lists.ipython.scipy.org/mailman/listinfo/ipython- > dev From glenn at tarbox.org Thu Mar 20 12:53:19 2008 From: glenn at tarbox.org (Glenn H Tarbox, PhD) Date: Thu, 20 Mar 2008 09:53:19 -0700 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <47e21ccf.0aaa660a.1579.1217@mx.google.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> Message-ID: <1206031999.3359.176.camel@ghtmyth.hq.tarbox.org> Ville responded to this as well but I'll add some On Thu, 2008-03-20 at 09:13 +0100, Laurent Dufr?chou wrote: > Hi all, > > Just to drop some more need on threading. > I've written a little piece of code to create a WX Ipython shell. > I've used for this a thread with an ipython0 instance and communicate with > the gui via polling (the first version) and now I'm starting to use > callback. > One big need I have and that cannot be resolved easily is to ba able to have > some sort of Ipython instance with GUI loop support + twisted support. > The idea: > > Any WXAPP/QTApp <-- XML RPC --> The twisted loop + Any GUI(QT/WX...) loop > + Ipython server. I guess the question is what you're trying to accomplish. So let me paste Ville's response which is exactly right and the core of any response to this kind of issue: "It really isn't, once we get the twisted reactor to be the "core" of IPython0. There is lots of fun to be had with this." So, what is being said here, and what lies at the core of the new / better / different IPython_x design is an event based architecture underlying whatever front ends we have. Embrace giving "the thread" away.... So, first a claim which should set the stage for a flame war: >>>>>>>>>>>> Soapbox Warning: <<<<<<<<<<<<<<<<< Threads are, for virtually all intents and purposes, BAD and should be avoided like the plague. (never had the plague... maybe its not as bad as threads) Of course, the above is wrong, but its way more right than any counter argument which will be brought forth. Now, when threads are necessary, they're critical and we can't get by without them. But realize, Python has 1 thread... thats right kids, regardless of the number of cores on your computer, python is only executing one actual instruction at a time. Of course, there's a lot of support the OS provides to your performance experience with another core such as IO etc... and some libraries do spawn threads underneath... but python itself is single "true thread"ed which is why you see words like GIL used and which is another huge issue in the broader python community and has caused any number of blog / mail group thread wars as python strives to deal with this multi-core world. Whats even more important is that threads are almost always unnecessary and are used, mostly, because people don't know how to accomplish certain tasks (blocking) without them. They require resource synchronization, module segmentation (or other synchronization migigation strategy) but at least they introduce non-deternimism, lower performance, and a general impossibility to debugging the intermittent failures you're sure to experience. (guido, the man himself writes: "Unfortunately, for most mortals, thread programming is just Too Hard to get right.... Even in Python -- every time someone gets into serious thread programming, they send me tons of bug reports, and half of them are subtle bugs in the Python interpreter, half of them are subtle problems in their own understanding of the consequences of multiple threads...." now lets add that virtually all the gui loops (and this holds true for most callback style event handling mechanisms) aren't thread safe... so, you need to go through some kind of prophylactic (always practice safe threading) when one thread touches something owned by another thread <<<<<<<<<<< steps off soap box >>>>>>>>>>>>>>>>>>> whew, feel better now. OK, so lets get back to what you were asking... so, what you really want to do is integrate various event models with pre-existing event handling mechanisms... and away we go.... So, you're using Wx. Unfortunately, it looks like you're in trouble to start with although I haven't been able to run this all the ground. According to some who should know, Wx behaves badly as an event engine... (apparently stops emitting timers / IO events during modal windows etc) so the current Twisted Wx Reactor, therefore uses threads to get around this. This strategy is mostly, why the twisted Wx reactor doesn't completely work right now... nothing inherently wrong with the design just some complexity associated with the thread handoffs which uses a generator and I haven't completely gotten through this one yet... the Qt reactor didn't require this as it is well behaved... so I inverted the design of twisted, kinda... basically, I use Qt as the core OS integration mechanism and have it call me when certain things need to happpen... basically that means, simply, that IO which I own and timing / scheduling is serviced by a few callbacks I register with Qt. This is mostly what happens with the gtk reactor as well. So, your sketch using XML-RPC as an integration strategy makes sense if you're inter-process but, I think, might be overkill in the same process. Integrating multiple event loops is nasty because they all "help" by abstracting (hiding) the underlying OS mechanisms which emit events to the application layer. If we can get around this (as I did with the Qt reactor) we're better off. Furthermore, I wonder if your strategy will work in general even with threads... it all depends on what the other event loop does with the foundational wrapping of the event sources.... so, if you try, for example, to add your own socket and try and block on select, will that interfere with Wx? If you had Wx and Qt in the same process, where would the window events go? None of these core engines acknowledge the existence of any other engines... and would likely behave very badly with the introduction of such a virus in their process space. If you're in a different process, all this is an entirely different discussion and is more about event IO... none of which requires threading... Qt, Twisted, and Wx all provide very rich mechanisms to register callbacks for this kind of event handling. -glenn > > With XML RPC you can just make any action inside Ipython instance. > I want this to be able to make a WX(and QT) console able to launch any QT/Wx > windows and thus independent to the Console App. > > With this you can have a really really good dev env for graphical app. > Moreover designing such a server will be raally good for people who want an > ipython console, because with this all the wx/QT/... console code will share > the same ipython object that will save lot of work. > > Glenn, perhaps my idea is stupid, but do you thing we will have a lot of > performance related problem with this, do you see any issue as you've worked > a little on something like this?? > Perhaps it is a too big thing for a simple Ipython console... > > Laurent > > -----Message d'origine----- > De : ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De > la part de Ville M. Vainio > Envoy? : jeudi 20 mars 2008 07:20 > ? : Glenn H Tarbox, PhD > Cc : ipython-dev Mailing list > Objet : Re: [IPython-dev] Twisted reactor + IPython0 > > On Thu, Mar 20, 2008 at 7:41 AM, Glenn H Tarbox, PhD > wrote: > > > On Thu, 2008-03-20 at 07:33 +0200, Ville M. Vainio wrote: > > > On Thu, Mar 20, 2008 at 12:35 AM, Glenn H Tarbox, PhD > wrote: > > > > > > > This is where I started originally. There's a better fix which I > posted > > > > a while ago. The below uses polling which is unnecessary. By > modifying > > > > shell, you can turn the whole thing into a callback. > > > > > > The ASPN recipe works like the current GUI backends, so it seems to > > > "obviously work". > > > > it works, just spins cycles unnecessarily. If you look through this > > list you'll see the minor changes I made to Shell (posted on Feb 9). My > > code generates a callback when somethings needs to be serviced as > > opposed to having a high-frequency polling loop try and determine the > > same thing > > Goodie. Frankly, I probably missed the original discussion because I > thought of it was mostly related to IPython1 (though obviously I can't > remember why I missed it) ;-) > -- Glenn H. Tarbox, PhD glenn at tarbox.org From fperez.net at gmail.com Thu Mar 20 13:47:36 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 20 Mar 2008 10:47:36 -0700 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> Message-ID: On Thu, Mar 20, 2008 at 1:33 AM, Ville M. Vainio wrote: > On Thu, Mar 20, 2008 at 10:13 AM, Laurent Dufr?chou > wrote: > > > With this you can have a really really good dev env for graphical app. > > Moreover designing such a server will be raally good for people who want an > > ipython console, because with this all the wx/QT/... console code will share > > the same ipython object that will save lot of work. > > > > Glenn, perhaps my idea is stupid, but do you thing we will have a lot of > > performance related problem with this, do you see any issue as you've worked > > a little on something like this?? > > Perhaps it is a too big thing for a simple Ipython console... > > It really isn't, once we get the twisted reactor to be the "core" of > IPython0. There is lots of fun to be had with this. It would certainly be great to have this done in ip0 as an optional component: we could always tell our users: you have two options for gui support: - the threads-based hack we have, which mostly but not always works, but has no dependencies outside the stdlib and your own GUI toolkit. - a robust, solid, twisted-based one (if this really proves to be the case in the long run). The second option is one more dependency but if it proves to really do the right thing, in the end I'm sure most users would run with it. After both options have existed for a while and receive real-world testing on all backends/platforms, we can see if we decide to completely deprecate the first or keep it. In addition, a twisted-based design is very much in line with the ip1 architecture, so I'd love to see this done for all the GUI backends using twisted. Does it have reactors for Qt3/4, Wx and GTK that work on Windows, Linux and OSX? That's our support requirement currently (even if it's not perfect, all those do work and have real users). cheers, f From vivainio at gmail.com Thu Mar 20 14:30:27 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Thu, 20 Mar 2008 20:30:27 +0200 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> Message-ID: <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> On Thu, Mar 20, 2008 at 7:47 PM, Fernando Perez wrote: > It would certainly be great to have this done in ip0 as an optional > component: we could always tell our users: you have two options for > gui support: It's in trunk now. It's still a threads-based hack, in that ipython mainloop runs in a thread, but now the mainloop is a twisted reactor. See IPython/twshell.py. You can try it out by "python twshell.py". kbdint and Exit seem to work. I used the stuff from mtexp branch, so macros should work, and it should survive reactor death (5 sec timeout as before). Now, I gotta run, but someone might want to take a look at it. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ellisonbg.net at gmail.com Thu Mar 20 14:49:57 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 20 Mar 2008 12:49:57 -0600 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> Message-ID: <6ce0ac130803201149o4001e507w9b2274afa054be81@mail.gmail.com> Noooooo ;-). Please, don't start doing stuff like this without all of us discussing the best direction. The parallel computing stuff in IPython1 is designed to work with IPython0 and uses a very carefully crafted approach for integrating with the Twisted main loop. Doing something like this could easily break IPython1 for the many current users - even if it is optional, I don't want it in IPython0. More discussion is needed first. Sorry I don't have more time to participate in the discussion right now - if you are interested in how we are handling this in IPython1, please look at that code base. What ever ends up in IPython0 _must_ be compatible with that. Brian On Thu, Mar 20, 2008 at 12:30 PM, Ville M. Vainio wrote: > On Thu, Mar 20, 2008 at 7:47 PM, Fernando Perez wrote: > > > It would certainly be great to have this done in ip0 as an optional > > component: we could always tell our users: you have two options for > > gui support: > > It's in trunk now. It's still a threads-based hack, in that ipython > mainloop runs in a thread, but now the mainloop is a twisted reactor. > See IPython/twshell.py. You can try it out by "python twshell.py". > kbdint and Exit seem to work. > > I used the stuff from mtexp branch, so macros should work, and it > should survive reactor death (5 sec timeout as before). Now, I gotta > run, but someone might want to take a look at it. > > > -- > Ville M. Vainio - vivainio.googlepages.com > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' > _______________________________________________ > > > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Thu Mar 20 18:15:14 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 20 Mar 2008 16:15:14 -0600 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> Message-ID: <6ce0ac130803201515j7366f715t3de24dc585de8cc8@mail.gmail.com> > > It would certainly be great to have this done in ip0 as an optional > > component: we could always tell our users: you have two options for > > gui support: > > It's in trunk now. It's still a threads-based hack, in that ipython > mainloop runs in a thread, but now the mainloop is a twisted reactor. > See IPython/twshell.py. You can try it out by "python twshell.py". > kbdint and Exit seem to work. > > I used the stuff from mtexp branch, so macros should work, and it > should survive reactor death (5 sec timeout as before). Now, I gotta > run, but someone might want to take a look at it. I should provide a few more details about my earlier objections to having this in ipython0. One of the big problems we have dealt with in IPython1 is that we want/need to have networking clients that are based on Twisted. Thus, we need to have the Twisted reactor running in the IPython0 frontend. But that is the easy part. The challenge is that we need to put a blocking interface on top of Twisted deferred returning functions and methods. This took us a _very_ long time to figure out how to do. Here is what we are doing currently in ipython1 (when using it from ipython0): 1) We simply start the twisted reactor in a thread. 2) When we need to call a method/function that returns a deferred, we use blockingCallFromThread. This uses a Queue to block the users thread (where Ipython0 is running) until the deferred has fired in the other thread (the twisted one). There code for these things is here: http://bazaar.launchpad.net/~ipython/ipython/ipython1-dev/annotate/ellisonbg%40gmail.com-20080311024040-ctltp13xxrh9iba7?file_id=twistedutil.py-20080302203151-17poomviavsf2zjj-141 Or if you have the ipython1-dev branch, it is in: ipython/kernel/twistedutil.py >From our testing so far, this seems to provide a fairly robust way of running the reactor in ipython0. ...queue scary music... But, I am not sure how all of this works with the current threaded Ipython shells. I think most of it works, but it probably fails to handle signals properly. Also, I am not sure how everything would behave if we run a Twisted GUI reactor. Here is another big problem lurking on the horizon. Eventually, your average IPython session will be an IPython1 session and will thus have the Twisted reactor running in its main thread. We want to be able to write networking clients that will run in that context. That itself is not a problem. The problem is that we want to be able to have blocking network clients. This means somehow we need to figure out a way of blocking on the result of a deferred. With a single thread, this is impossible. The only way I know of doing this is to have two threads: 1) One running the twisted reactor 2) Another that runs the users code Then user code can use blockingCallFromThread to block on a deferred. While we don't yet have this implemented in Ipython1, it should not be difficult. The bigger complication comes in when you add a GUI event loop to the picture. Then you need to make sure that all three things (GUI event loop, twisted reactor and user code that potentially needs to block on a deferred) work together in a thread happy manner. That is the main thing to keep in mind. Sorry about my earlier rant. I don't want to discourage people from playing around with this stuff. But, this stuff is extremely complicated and easy to break. Please just keep in mind that currently we are in this really odd state where IPython1 needs to be able to work from within IPython0. Cheers, Brian > > -- > Ville M. Vainio - vivainio.googlepages.com > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' > _______________________________________________ > > > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From laurent.dufrechou at gmail.com Thu Mar 20 19:22:31 2008 From: laurent.dufrechou at gmail.com (=?windows-1250?Q?Laurent_Dufr=E9chou?=) Date: Fri, 21 Mar 2008 00:22:31 +0100 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <6ce0ac130803201515j7366f715t3de24dc585de8cc8@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> <6ce0ac130803201515j7366f715t3de24dc585de8cc8@mail.gmail.com> Message-ID: <47e2f1c4.0305560a.7758.5828@mx.google.com> Thanks for the reply guys. I must clarify my idea: My main objective is to provide a set of classes that anybody can use in QT/WX/whatelse to create easily some ipython console GUI widget (QTIpython,WXIpython,TKIpython etc...) My work is based on WX but can be easily used for QT or other.I'm using Scintilla for the display so... can be also easy to be ported. (will see this weekend with all the guy how I could interface with pyreadline) So,first I wrote a little piece of code to get ipython shell inside a wx widget. It was based on user keyboard interaction with no thread. I used for this a piece of gtk console and IPInteractive Shell class where I could launch some commands line by line. Then arise the problem of infinite loop: While 1: Pass With this my widget and all my GUI was frozen, using a thread where I sent line by line my command to an IPInteractive Shell solved this issue. That's why I needed thread glenn ;) A little Ascii drawing: Thread wich feeds Ipython instance line by line <--- communicate --> WxApp Main loop Thread Now I've got this problem: User want to be able to use Wx Ipython console widget with for example QT4 support or WX support... So I see no other idea to give them this class run in an other process where they could launch any GUI unrelated to my widget GUI loop. That's why I wanted xml-rpc and twisted support in ipython0. (If you take a look at this it looks like that ipython1 should be the WAY to do, but I don't know if ipython1 has interactive support like ipython0 and ipython0 NEEDS GUI widgets ;) ) So perhaps twisted integration in ipython0 for that is not a clever idea, perhaps a simpler xmlrpc server loop can be OK. I really want to have GUI loop+XML/RPC loop+ipython interactivity in a class that I could call to do all of this. Guys,what is the best way? Ville: I'll take a look at your file this week end, when gael and others guys in france will have explained me all the bzr stuff :) Glenn: >So, your sketch using XML-RPC as an integration strategy makes sense if you're inter-process but, I think, might be overkill in the same process. Integrating multiple event loops is nasty because they >all "help" by abstracting (hiding) the underlying OS mechanisms which emit events to the application layer. If we can get around this (as I did with the Qt reactor) we're better off. In fact xml-rpc should be used when guys want to play with GUI in Ipython shell widget whithout burry my WXapp. If it not a required thing than, xml-rpc is unusefull sure :), but I also don't want to split my code... >Furthermore, I wonder if your strategy will work in general even with threads... it all depends on what the other event loop does with the foundational wrapping of the event sources.... so, if you try, >for example, to add your own socket and try and block on select, will that interfere with Wx? Well, for what I've tried on the current code, it works ;) I mean the ipython instance with no GUI support and a wx loop controlling it (my wxConsole widget) > If you had Wx and Qt in the same process, where would the window events go? None of these core engines >acknowledge the existence of any other engines... and would likely behave very badly with the introduction of such a virus in their process space. That why I want to do the "xml-rpc/ipython0 in an other process" trick :) >If you're in a different process, all this is an entirely different discussion and is more about event IO... none of which requires threading... Qt, Twisted, and Wx all provide very rich mechanisms to >register callbacks for this kind of event handling. Yep I discovered that, before my wx widget was doing some polling on on the the ipython thread, now I'm using wx callback and no more polling is necessary, but the thread is still necessary due to while 1: pass lockup. In a different process n omore thread is necessary and that is what I want :) Glenn, if I well understood, what you've done is QT support and twisted and ipython inside the "same loop"? It could be interesting to see if we could have some sort of unified class to support the whole thing wx+twisted+ipython and qt+twisted+ipython. We need to check also Brian warnings :) Brian: Do you have a better idea than twisted integration for my specific need (that will less destroy ipython1)??? Laurent No virus found in this outgoing message. Checked by AVG. Version: 7.5.519 / Virus Database: 269.21.7/1335 - Release Date: 19/03/2008 09:54 From gael.varoquaux at normalesup.org Fri Mar 21 02:59:09 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 21 Mar 2008 07:59:09 +0100 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <47e2f1c4.0305560a.7758.5828@mx.google.com> References: <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> <6ce0ac130803201515j7366f715t3de24dc585de8cc8@mail.gmail.com> <47e2f1c4.0305560a.7758.5828@mx.google.com> Message-ID: <20080321065909.GC25208@phare.normalesup.org> On Fri, Mar 21, 2008 at 12:22:31AM +0100, Laurent Dufr?chou wrote: > Ville: > I'll take a look at your file this week end, when gael and others guys in > france will have explained me all the bzr stuff :) You need to install bzr (under Linux it should be trivial, under windows, it is just a Python-based program). Once you have it, I made a small note for the nipy development, that could be useful to you: http://bazaar.launchpad.net/~nipy-developers/nipy/trunk/annotate/cburns%40berkeley.edu-20080320145517-20ap9pirmqr1no0k?file_id=svn2bzr-006188684113f68f12549ab5c42381681e5b61fc HTH, Ga?l From vivainio at gmail.com Fri Mar 21 08:57:30 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Fri, 21 Mar 2008 14:57:30 +0200 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <6ce0ac130803201149o4001e507w9b2274afa054be81@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205966116.26877.104.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> <6ce0ac130803201149o4001e507w9b2274afa054be81@mail.gmail.com> Message-ID: <46cb515a0803210557u2d453a90h49adafe751a7518@mail.gmail.com> On Thu, Mar 20, 2008 at 8:49 PM, Brian Granger wrote: > we are handling this in IPython1, please look at that code base. What > ever ends up in IPython0 _must_ be compatible with that. The stuff in IPython0 is not in the core, so this is not a problem (the same reason as explained before - we can't have a twisted dependency). -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ellisonbg.net at gmail.com Fri Mar 21 09:24:37 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 21 Mar 2008 07:24:37 -0600 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <46cb515a0803210557u2d453a90h49adafe751a7518@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> <6ce0ac130803201149o4001e507w9b2274afa054be81@mail.gmail.com> <46cb515a0803210557u2d453a90h49adafe751a7518@mail.gmail.com> Message-ID: <6ce0ac130803210624v7ee1a7c6u2365ac23de076333@mail.gmail.com> On Fri, Mar 21, 2008 at 6:57 AM, Ville M. Vainio wrote: > On Thu, Mar 20, 2008 at 8:49 PM, Brian Granger wrote: > > > we are handling this in IPython1, please look at that code base. What > > ever ends up in IPython0 _must_ be compatible with that. > > The stuff in IPython0 is not in the core, so this is not a problem > (the same reason as explained before - we can't have a twisted > dependency). Absolutely true. An optional Twisted dep for IPython0 would be just fine. Brian > -- > > > Ville M. Vainio - vivainio.googlepages.com > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' > From ellisonbg.net at gmail.com Fri Mar 21 09:28:02 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 21 Mar 2008 07:28:02 -0600 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <6ce0ac130803210624v7ee1a7c6u2365ac23de076333@mail.gmail.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> <6ce0ac130803201149o4001e507w9b2274afa054be81@mail.gmail.com> <46cb515a0803210557u2d453a90h49adafe751a7518@mail.gmail.com> <6ce0ac130803210624v7ee1a7c6u2365ac23de076333@mail.gmail.com> Message-ID: <6ce0ac130803210628s1bcdae41tf26a241aa2110174@mail.gmail.com> > Absolutely true. An optional Twisted dep for IPython0 would be just fine. I just realized that this may be confusing. Just in terms of dependencies, having IPython0 depend on Twisted (optionally of course) is not a problem. But, how IPython0 _uses_ twisted (even though optional) should be compatible with the way people are currently using Ipython1 for parallel computing - they simply use it with IPython0. That is, the any IPython0 Twisted usage must not break the IPython0+IPython1 usage pattern that is most common today. Brian > Brian > > > -- > > > > > > > > Ville M. Vainio - vivainio.googlepages.com > > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' > > > From ellisonbg.net at gmail.com Fri Mar 21 16:09:59 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 21 Mar 2008 14:09:59 -0600 Subject: [IPython-dev] Twisted reactor + IPython0 In-Reply-To: <47e2f1c4.0305560a.7758.5828@mx.google.com> References: <46cb515a0803191439g6b0e7115r28a6c68d96a66a0c@mail.gmail.com> <46cb515a0803192233l68140a5fje1b167710d7ff331@mail.gmail.com> <1205991678.3359.8.camel@ghtmyth.hq.tarbox.org> <46cb515a0803192320w22c5370bsb7b16813377e0a98@mail.gmail.com> <47e21ccf.0aaa660a.1579.1217@mx.google.com> <46cb515a0803200133m429d0a92x566878f63d789897@mail.gmail.com> <46cb515a0803201130j648b9b79qf5601194bff98b0e@mail.gmail.com> <6ce0ac130803201515j7366f715t3de24dc585de8cc8@mail.gmail.com> <47e2f1c4.0305560a.7758.5828@mx.google.com> Message-ID: <6ce0ac130803211309s1036581frba09fc9aa273cbb2@mail.gmail.com> > I must clarify my idea: > My main objective is to provide a set of classes that anybody can use in > QT/WX/whatelse to create easily some ipython console GUI widget > (QTIpython,WXIpython,TKIpython etc...) Great! > My work is based on WX but can be easily used for QT or other.I'm using > Scintilla for the display so... can be also easy to be ported. > (will see this weekend with all the guy how I could interface with > pyreadline) > > So,first I wrote a little piece of code to get ipython shell inside a wx > widget. > It was based on user keyboard interaction with no thread. > I used for this a piece of gtk console and IPInteractive Shell class where I > could launch some commands line by line. > Then arise the problem of infinite loop: > While 1: > Pass > With this my widget and all my GUI was frozen, using a thread where I sent > line by line my command to an IPInteractive Shell solved this issue. > That's why I needed thread glenn ;) > > A little Ascii drawing: > > Thread wich feeds Ipython instance line by line <--- communicate --> WxApp > Main loop Thread > > Now I've got this problem: > User want to be able to use Wx Ipython console widget with for example QT4 > support or WX support... > So I see no other idea to give them this class run in an other process where > they could launch any GUI unrelated to my widget GUI loop. > That's why I wanted xml-rpc and twisted support in ipython0. > (If you take a look at this it looks like that ipython1 should be the WAY to > do, but I don't know if ipython1 has interactive support like ipython0 and > ipython0 NEEDS GUI widgets ;) ) > So perhaps twisted integration in ipython0 for that is not a clever idea, > perhaps a simpler xmlrpc server loop can be OK. > I really want to have GUI loop+XML/RPC loop+ipython interactivity in a class > that I could call to do all of this. > > Guys,what is the best way? It really depends on your goals. If you need something super quick and are OK hacking together something with IPython0, then just do that. But just a warning. You will end up hacking something together that only half works and as IPython1 matures, you will end up re-writing it to use IPython1. IPython1 is being designed from the ground up to support this type of thing. Does IPython1 have interactive support - absolutely!!! - overwise it would have the "I" in front. You could start developing such GUI widgets today using IPython1 and the result would be robust, well designed code that will last for years. Example: Just last week, Barry Wark created a very nice Cocoa App for IPython1 using PyObjC. While the resulting app doesn't have all of the features of IPython0 (magics, tab completion, shell commands, etc.), those are the easy part. The important and hard part is getting the whole design right. And that is already done in IPython1. Depending on how you want to design the GUI stuff, you might need to learn a bit about twisted, but that is not too bad. The key point is that if you go this route, you will end up having fun coding things that are well designed and will stand the test of time, rather than hacking around trying to square a circle. We are more than willing to point you to the relevant code in IPython1. > Ville: > I'll take a look at your file this week end, when gael and others guys in > france will have explained me all the bzr stuff :) Sounds fun! > Glenn: > > > >So, your sketch using XML-RPC as an integration strategy makes sense if > you're inter-process but, I think, might be overkill in the same process. > Integrating multiple event loops is nasty because they >all "help" by > abstracting (hiding) the underlying OS mechanisms which emit events to the > application layer. If we can get around this (as I did with the Qt reactor) > we're better off. > > In fact xml-rpc should be used when guys want to play with GUI in Ipython > shell widget whithout burry my WXapp. > If it not a required thing than, xml-rpc is unusefull sure :), but I also > don't want to split my code... > > > >Furthermore, I wonder if your strategy will work in general even with > threads... it all depends on what the other event loop does with the > foundational wrapping of the event sources.... so, if you try, >for example, > to add your own socket and try and block on select, will that interfere with > Wx? > > Well, for what I've tried on the current code, it works ;) I mean the > ipython instance with no GUI support and a wx loop controlling it (my > wxConsole widget) > > > > If you had Wx and Qt in the same process, where would the window events > go? None of these core engines >acknowledge the existence of any other > engines... and would likely behave very badly with the introduction of such > a virus in their process space. Yes, multiple GUIs in the same process is a bad idea. > That why I want to do the "xml-rpc/ipython0 in an other process" trick :) > > > >If you're in a different process, all this is an entirely different > discussion and is more about event IO... none of which requires threading... > Qt, Twisted, and Wx all provide very rich mechanisms to >register callbacks > for this kind of event handling. > > Yep I discovered that, before my wx widget was doing some polling on on the > the ipython thread, now I'm using wx callback and no more polling is > necessary, but the thread is still necessary due to while 1: pass lockup. In > a different process n omore thread is necessary and that is what I want :) > > Glenn, if I well understood, what you've done is QT support and twisted and > ipython inside the "same loop"? > It could be interesting to see if we could have some sort of unified class > to support the whole thing wx+twisted+ipython and qt+twisted+ipython. > We need to check also Brian warnings :) > > Brian: > Do you have a better idea than twisted integration for my specific need > (that will less destroy ipython1)??? If you want to got this route (using twisted in ipython0) there is a good route: 1) Start the reactor in a thread 2) Use blockingCallFromThread to call Twisted using methods. blockingCallFromThread allows you to block in one thread until a deferred fires in the Twisted thread. You can see how we are doing this by checking out ipython1 and looking in: ipython1/kernel/twistedutil.py ipython1/kernel/client.py ipython1/kernel/multiengineclient.py Cheers, Brian > Laurent > > > No virus found in this outgoing message. > Checked by AVG. > Version: 7.5.519 / Virus Database: 269.21.7/1335 - Release Date: 19/03/2008 > 09:54 > > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Fri Mar 21 16:25:41 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 21 Mar 2008 14:25:41 -0600 Subject: [IPython-dev] Discussion on Bruce Eckel's blog In-Reply-To: <1206022514.3359.71.camel@ghtmyth.hq.tarbox.org> References: <1206022514.3359.71.camel@ghtmyth.hq.tarbox.org> Message-ID: <6ce0ac130803211325y42205594y24561c9cd4b57dec@mail.gmail.com> > Is he trying to discuss the issues of parallelism, threading, co-routine > abstractions, inter process-communication? In the same brief blurb he > discusses and mixes all four. Makes mention of the GIL and how java > multithreading through co-routines (stackless???) are... what? I got > confused even parsing the words. none of these issues are related at > the level of the post. > Stack frames, threads, processes, IPC and netowrking are, clearly, > related but wildly different subjects... not to mention that none of > these have anything to do with solving parallel computing problems other > than we might want to use multiple computers and electricity... I completely agree with this evaluation. Parallel and distributed computing is a very complex and rich field and throwing all of these things together really confuses the problems and solutions. > And IPython1? Sure, there's a notion of a redesign to embrace actual > parallel computation issues, and even a couplea examples of how one > might use asynchronous models to implement map-reduce (or scatter-gather > or whatever) > > But the IPython1 effort is mostly about embracing the asynchronous model > (read Twisted) with a sugary sweet layer of IPython to make it a bit > less painful. Unless I missed a bunch of documentation somewhere > (certainly possible given the state of the documentation), this hardly > rises to: "a powerful architecture for parallel computing," So, I must confess that this is language that I am responsible for, so it is not something that Bruce can take the blame on. But, I would argue that the statement is true. At the same time, I admit that we (mostly myself) have done a really bad job of laying out why this is the case. This is evidenced by the fact that Parallel Processing and Processing are being spoken of as the leading solutions in this area - even though IPython1 is more mature and technically superior. I should say though that I really like both Parallel Processing and Processing - I just think we are trying to accomplish something much more ambitious with IPython1 and are having good successes with our efforts. > IPython1 has a notion > that there should be an architecture and even some early examples... but > there's no new model, no power other than doing stuff correctly with > modern tools. I like this thinking, even though I don't fully agree. The ironic thing is that "doing stuff correctly with modern tools" is extremely difficult and uncommon when it comes to parallel and distributed computing. Where I disagree slightly is that I think you do get something really powerful when you 1) have a well designed architecture and 2) do things correctly with modern tools. Cheers, Brian From steve at shrogers.com Fri Mar 21 22:17:22 2008 From: steve at shrogers.com (Steven H. Rogers) Date: Fri, 21 Mar 2008 20:17:22 -0600 Subject: [IPython-dev] Discussion on Bruce Eckel's blog In-Reply-To: <6ce0ac130803211325y42205594y24561c9cd4b57dec@mail.gmail.com> References: <1206022514.3359.71.camel@ghtmyth.hq.tarbox.org> <6ce0ac130803211325y42205594y24561c9cd4b57dec@mail.gmail.com> Message-ID: <47E46C32.1040006@shrogers.com> Brian Granger wrote: >> Stack frames, threads, processes, IPC and netowrking are, clearly, >> related but wildly different subjects... not to mention that none of >> these have anything to do with solving parallel computing problems other >> than we might want to use multiple computers and electricity... >> > > I completely agree with this evaluation. Parallel and distributed > computing is a very complex and rich field and throwing all of these > things together really confuses the problems and solutions. > ... > Bruce seems to be a bit confused, but I think this represents the state of most programmers w.r.t. concurrent, parallel, and distributed processing. They want to continue programming the way they're accustomed to and just have it magically take advantage of parallel and/or distributed hardware. They're not interested in, and don't really want to know about, the technical difficulties. > ... This is evidenced by the fact that Parallel Processing and > Processing are being spoken of as the leading solutions in this area - > even though IPython1 is more mature and technically superior. I > should say though that I really like both Parallel Processing and > Processing - I just think we are trying to accomplish something much > more ambitious with IPython1 and are having good successes with our > efforts Those successes need to be communicated. I believe a PEP in in preparation to include some sort of parallel solution into the Python Standard Library. # Steve From fperez.net at gmail.com Sat Mar 22 10:45:43 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 22 Mar 2008 07:45:43 -0700 Subject: [IPython-dev] #ipython on IRC (freenode server) Message-ID: Hi all, we've joined the (existing, but seldom used) #ipython channel on freenode. I know we've been doing a poor job of keeping the rest of the wider user/developer community up to date with the development plans and work on ipython, but we've also promised to do better. This is the first (small) step in that direction: for the next two days, Stefan van der Walt, Gael Varoquaux, Laurent Dufrechou (who recently contributed the Wx shell) and myself are hanging out at Gael's parent's house south of Paris, working on kickstarting the integration work for ip0/ip1, building on top of all of the recent flurry of activity. We'll also gradually document better the development process, figure out what to do about Trac and old tickets, and hopefully in a few months we'll not only have more functioning code, but also a more active community that really encourages *anyone* who is interested in the project and willing to participate, to do so. Cheers, f From ondrej at certik.cz Sat Mar 22 23:41:33 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Sun, 23 Mar 2008 04:41:33 +0100 Subject: [IPython-dev] sphinx documentation for ipython1 Message-ID: <85b5c3130803222041mbb83d34x397c6dda646d0778@mail.gmail.com> Hi, I committed a nice sphinx based documentation to the ipython1-doc branch. You can try it out for example by: $ bzr branch http://bazaar.launchpad.net/~ipython/ipython/ipython1-doc $ cd ipython1-doc/docs $ make html $ epiphany _build/html/index.html (Use your favourite browser instead of epiphany). The "make html" will download Sphinx from the net (so you need to have an internet connection) and puts it into the sphinx directory in the current dir. Subsequent invocations of "make html" will reuse it. I suggest you to browse the history of my commits to see what exact changes I had to do in order to make all this work. The changes broke the pdf generation - this would have to be fixed, but I suggest to get at least the html documentation right and polished, only then try to fix the pdf generation. Also the Sphinx framework should be able to generate pdf files too. Unfortunately it took my almost the whole night to fix all things (it's 4:40am), so I won't be able to help much with ipython on Sunday, as I will have to work on other things. But I'll be available if you need to explain anything regarding my patches. Enjoy hacking at Gaels, Ondrej From gael.varoquaux at normalesup.org Sun Mar 23 06:13:13 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sun, 23 Mar 2008 11:13:13 +0100 Subject: [IPython-dev] sphinx documentation for ipython1 In-Reply-To: <85b5c3130803222041mbb83d34x397c6dda646d0778@mail.gmail.com> References: <85b5c3130803222041mbb83d34x397c6dda646d0778@mail.gmail.com> Message-ID: <20080323101313.GA14881@phare.normalesup.org> On Sun, Mar 23, 2008 at 04:41:33AM +0100, Ondrej Certik wrote: > I committed a nice sphinx based documentation to the ipython1-doc > branch. You can try it out for example by: > $ bzr branch http://bazaar.launchpad.net/~ipython/ipython/ipython1-doc > $ cd ipython1-doc/docs > $ make html > $ epiphany _build/html/index.html > (Use your favourite browser instead of epiphany). Very sweet. Nice ! Ga?l From fperez.net at gmail.com Sun Mar 23 06:49:44 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 23 Mar 2008 03:49:44 -0700 Subject: [IPython-dev] sphinx documentation for ipython1 In-Reply-To: <85b5c3130803222041mbb83d34x397c6dda646d0778@mail.gmail.com> References: <85b5c3130803222041mbb83d34x397c6dda646d0778@mail.gmail.com> Message-ID: On Sat, Mar 22, 2008 at 8:41 PM, Ondrej Certik wrote: > Hi, > > I committed a nice sphinx based documentation to the ipython1-doc > branch. You can try it out for example by: > > $ bzr branch http://bazaar.launchpad.net/~ipython/ipython/ipython1-doc > $ cd ipython1-doc/docs > $ make html > $ epiphany _build/html/index.html > > (Use your favourite browser instead of epiphany). Sweet! The built-in search (jscript tricks, I presume) is absolutely phenomenal. I'm updating the cron job to pull from this so everyone has access to these docs, we should merge this very soon into the dev branch. > The "make html" will download Sphinx from the net (so you need to have > an internet connection) and puts it into the sphinx directory in the > current dir. Subsequent invocations of "make html" will reuse it. I > suggest you to browse the history of my commits to see what exact > changes I had to do in order to make all this work. > > The changes broke the pdf generation - this would have to be fixed, > but I suggest to get at least the html documentation right and > polished, only then try to fix the pdf generation. > Also the Sphinx framework should be able to generate pdf files too. > > Unfortunately it took my almost the whole night to fix all things > (it's 4:40am), so I won't be able to help much with ipython on Sunday, > as I will have to work on other things. But I'll be available if you > need to explain anything regarding my patches. Thanks so much! This is great, I hope you can get some rest, and don't worry for now. Let me know when you have a minute to look at the xml2rst business for the other docs, since that would be the remaining piece missing. > > Enjoy hacking at Gaels, We certainly are: the hospitality given by him and his parents has been amazing. Chalk one to great open source development sponsorship! Cheers, f From fperez.net at gmail.com Sun Mar 23 19:17:10 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 23 Mar 2008 16:17:10 -0700 Subject: [IPython-dev] Docs for ipython1 Message-ID: Hi all, just to ping those who might not have been hanging on IRC, thanks to Ondrej's work (completing what was started on the previous meeting at Enthought), we now have: http://ipython.scipy.org/doc/ipython1/html/ http://ipython.scipy.org/doc/ipython1/ipython1.pdf these are the docs for ipython1 in html and pdf, using sphinx, the Python 2.6 official documentation system. These docs are guaranteed to be up to date, since they get built every night from the main bzr branch. As I recently said, I know we've done a poor job of recently communicating our development effort and keeping everyone informed, involved and welcome. I hope that better docs and this weekend's interaction on IRC are proof that we mean it when we say that we're trying to do better. I'd like to mention that while sprinting on IRC, when we decide to jump in on skype for a direct discussion *everyone* is welcome! Skype allows fairly large conference calls, and we'd love to have others on board. We don't yet have funds for large face to face ipython meetings (except when piggybacking on other projects), so irc+skype will have to do. But anyone is welcome to participate if we switch over to skype briefly, so please do NOT interpret such switches on IRC as a 'closing of the doors to the cognoscenti'. It is just a practical tool to talk rapidly about a given topic beyond what typing on IRC allows. Cheers, f From ondrej at certik.cz Sun Mar 23 21:41:54 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Mon, 24 Mar 2008 02:41:54 +0100 Subject: [IPython-dev] Docs for ipython1 In-Reply-To: References: Message-ID: <85b5c3130803231841y45346051yb056a333c7e5a766@mail.gmail.com> On Mon, Mar 24, 2008 at 12:17 AM, Fernando Perez wrote: > Hi all, > > just to ping those who might not have been hanging on IRC, thanks to > Ondrej's work (completing what was started on the previous meeting at > Enthought), we now have: > > http://ipython.scipy.org/doc/ipython1/html/ > http://ipython.scipy.org/doc/ipython1/ipython1.pdf > > these are the docs for ipython1 in html and pdf, using sphinx, the > Python 2.6 official documentation system. These docs are guaranteed > to be up to date, since they get built every night from the main bzr > branch. You meantioned that you'd like to have the search box - so I asked upstream and Georg fixed it in the Sphinx svn, so if you download the svn, you'll get a search box on every page. Ondrej From vivainio at gmail.com Mon Mar 24 05:14:57 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 24 Mar 2008 11:14:57 +0200 Subject: [IPython-dev] IPython winpdb interface In-Reply-To: <47E6E860.30702@winpdb.org> References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com> <47E6E860.30702@winpdb.org> Message-ID: <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com> [CC-ing to ipython-dev, about adding winpdb support to IPython] On Mon, Mar 24, 2008 at 1:31 AM, Nir wrote: > Lets start with solving your initial scenario. > Can you describe how is such an ipython session supposed to look like? If you now execute %run -d test.py IPython now sets the breakpoint at start of test.py and launches it in pdb, running the test.py in ipython process. I'd like to add a new magic command %wpdb (in an extension, say ipy_winpdb), so you can do %wpdb test.py which will do the same thing, i.e. launch test.py in the ipython process, with rpdb2 breakpoint at start of the script. The rpdb2 should only be started once in the IPython process. Likewise, if the ipy_winpdb extension is loaded and post-mortem debugging is enabled, hitting an exception would break rpdb2 at the exception. All of this is pretty easy in ipython side, we only need API support in rpdb2 module to create a breakpoint at first valid position in a file on file system. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Mon Mar 24 05:30:24 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 24 Mar 2008 11:30:24 +0200 Subject: [IPython-dev] Docs for ipython1 In-Reply-To: References: Message-ID: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> On Mon, Mar 24, 2008 at 1:17 AM, Fernando Perez wrote: > http://ipython.scipy.org/doc/ipython1/html/ > http://ipython.scipy.org/doc/ipython1/ipython1.pdf > > these are the docs for ipython1 in html and pdf, using sphinx, the > Python 2.6 official documentation system. These docs are guaranteed And aren't they pretty! There still seems to be some reST artifacts in code examples e.g. here: http://ipython.scipy.org/doc/ipython1/html/parallel_task.html#quickstart-task-farming Ondrej, any idea when you could have something like this up for IPython0 as well? -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Mon Mar 24 05:35:50 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 24 Mar 2008 02:35:50 -0700 Subject: [IPython-dev] Sprint over, offline... Message-ID: Hi all, I just wanted to briefly thank Gael, AND his family, for the great hospitality during the sprint. We got a lot done, and I hope this will kickstart a lot of things that had been stalling for a while. I have to run now and I'll be offline until the end of the week, and I think so will Gael and Stefan, but we'll do a little recap later. Thanks to everyone who participated via irc/skype, and see you all soon! Cheers, f From ondrej at certik.cz Mon Mar 24 08:50:34 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Mon, 24 Mar 2008 13:50:34 +0100 Subject: [IPython-dev] Docs for ipython1 In-Reply-To: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> Message-ID: <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> On Mon, Mar 24, 2008 at 10:30 AM, Ville M. Vainio wrote: > On Mon, Mar 24, 2008 at 1:17 AM, Fernando Perez wrote: > > > http://ipython.scipy.org/doc/ipython1/html/ > > http://ipython.scipy.org/doc/ipython1/ipython1.pdf > > > > these are the docs for ipython1 in html and pdf, using sphinx, the > > Python 2.6 official documentation system. These docs are guaranteed > > And aren't they pretty! > > There still seems to be some reST artifacts in code examples e.g. here: > > http://ipython.scipy.org/doc/ipython1/html/parallel_task.html#quickstart-task-farming We had a similar problem in sympy with different artefacts. Please report them to Georg on the doc-sig mailinglist. He fixed our problems in about 2 hours. > Ondrej, any idea when you could have something like this up for > IPython0 as well? The xml docs need to be converted, but I won't have time for this anytime soon unfortunately. But I just updated the ipython1 docs to the latest (fixed) Sphinx, so now the search box appears on every page. Ondrej From ellisonbg.net at gmail.com Mon Mar 24 12:11:58 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Mon, 24 Mar 2008 10:11:58 -0600 Subject: [IPython-dev] Docs for ipython1 In-Reply-To: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> Message-ID: <6ce0ac130803240911h7a27e4beh28ba75f78d4d4d2d@mail.gmail.com> > There still seems to be some reST artifacts in code examples e.g. here: > > http://ipython.scipy.org/doc/ipython1/html/parallel_task.html#quickstart-task-farming Where are the artifacts exactly? I looked over that section and couldn't see any obvious ones. Brian From vivainio at gmail.com Mon Mar 24 12:16:22 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 24 Mar 2008 18:16:22 +0200 Subject: [IPython-dev] Docs for ipython1 In-Reply-To: <6ce0ac130803240911h7a27e4beh28ba75f78d4d4d2d@mail.gmail.com> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <6ce0ac130803240911h7a27e4beh28ba75f78d4d4d2d@mail.gmail.com> Message-ID: <46cb515a0803240916r52701d26jb52efe529e9117ef@mail.gmail.com> On Mon, Mar 24, 2008 at 6:11 PM, Brian Granger wrote: > > There still seems to be some reST artifacts in code examples e.g. here: > > > > http://ipython.scipy.org/doc/ipython1/html/parallel_task.html#quickstart-task-farming > > Where are the artifacts exactly? I looked over that section and > couldn't see any obvious ones. Stuff like this: In [8]: tr = ``Task``_results[73] In [9]: tr Out[9]: ``TaskResult``[ID:73]:{'a':73} -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ellisonbg.net at gmail.com Mon Mar 24 12:42:25 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Mon, 24 Mar 2008 10:42:25 -0600 Subject: [IPython-dev] Docs for ipython1 In-Reply-To: <46cb515a0803240916r52701d26jb52efe529e9117ef@mail.gmail.com> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <6ce0ac130803240911h7a27e4beh28ba75f78d4d4d2d@mail.gmail.com> <46cb515a0803240916r52701d26jb52efe529e9117ef@mail.gmail.com> Message-ID: <6ce0ac130803240942n589d7d82g8ce630a81ee21cba@mail.gmail.com> Ooops, I will follow this up. Thanks Brian On Mon, Mar 24, 2008 at 10:16 AM, Ville M. Vainio wrote: > > On Mon, Mar 24, 2008 at 6:11 PM, Brian Granger wrote: > > > There still seems to be some reST artifacts in code examples e.g. here: > > > > > > http://ipython.scipy.org/doc/ipython1/html/parallel_task.html#quickstart-task-farming > > > > Where are the artifacts exactly? I looked over that section and > > couldn't see any obvious ones. > > Stuff like this: > > In [8]: tr = ``Task``_results[73] > > In [9]: tr > Out[9]: ``TaskResult``[ID:73]:{'a':73} > > > > > -- > Ville M. Vainio - vivainio.googlepages.com > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' > From laurent.dufrechou at gmail.com Mon Mar 24 12:42:36 2008 From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=) Date: Mon, 24 Mar 2008 17:42:36 +0100 Subject: [IPython-dev] IPython winpdb interface In-Reply-To: <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com> References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com> <47E6E860.30702@winpdb.org> <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com> Message-ID: <47e7da01.0407560a.53ef.0285@mx.google.com> Hi ville, That's a GOOD idea :) I'm used to debug my script with winpdb crossplatform debugger and I'm quite happy to listen this :) Does the %wpdb will: - launch the winpdb GUI or - only the server and we will have to launch the winpdb gui and attach to the server manually or - usage is only via rdpb server without winpdb GUI? What are your plan about the usage scenario?? -----Message d'origine----- De?: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De la part de Ville M. Vainio Envoy??: lundi 24 mars 2008 10:15 ??: Nir; ipython-dev Mailing list Objet?: Re: [IPython-dev] IPython winpdb interface [CC-ing to ipython-dev, about adding winpdb support to IPython] On Mon, Mar 24, 2008 at 1:31 AM, Nir wrote: > Lets start with solving your initial scenario. > Can you describe how is such an ipython session supposed to look like? If you now execute %run -d test.py IPython now sets the breakpoint at start of test.py and launches it in pdb, running the test.py in ipython process. I'd like to add a new magic command %wpdb (in an extension, say ipy_winpdb), so you can do %wpdb test.py which will do the same thing, i.e. launch test.py in the ipython process, with rpdb2 breakpoint at start of the script. The rpdb2 should only be started once in the IPython process. Likewise, if the ipy_winpdb extension is loaded and post-mortem debugging is enabled, hitting an exception would break rpdb2 at the exception. All of this is pretty easy in ipython side, we only need API support in rpdb2 module to create a breakpoint at first valid position in a file on file system. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' _______________________________________________ IPython-dev mailing list IPython-dev at scipy.org http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev No virus found in this incoming message. Checked by AVG. Version: 7.5.519 / Virus Database: 269.21.8/1340 - Release Date: 23/03/2008 18:50 No virus found in this outgoing message. Checked by AVG. Version: 7.5.519 / Virus Database: 269.21.8/1340 - Release Date: 23/03/2008 18:50 From fperez.net at gmail.com Mon Mar 24 18:58:56 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 24 Mar 2008 15:58:56 -0700 Subject: [IPython-dev] Docs for ipython1 In-Reply-To: <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> Message-ID: On Mon, Mar 24, 2008 at 5:50 AM, Ondrej Certik wrote: > The xml docs need to be converted, but I won't have time for this > anytime soon unfortunately. But I just updated the ipython1 docs to > the latest (fixed) Sphinx, so now the search box appears on every > page. That's great (the search box)! I just ran the update by hand and now the search box is now always visible on the side. This is great!!! And don't worry about ip0 for now, if you can help us later that's great, but you've already done a lot! Cheers, f From laurent.dufrechou at gmail.com Mon Mar 24 19:14:43 2008 From: laurent.dufrechou at gmail.com (=?windows-1250?Q?Laurent_Dufr=E9chou?=) Date: Tue, 25 Mar 2008 00:14:43 +0100 Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk In-Reply-To: References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> Message-ID: <47e835ea.0c07560a.5adf.ffffb9c5@mx.google.com> Hi all, Where can I put some info on how to install bzr under Vista? I've installed it with the .exe that is a big error and nobody must do that. So I would like to warn newcomers how to set up bzr + bzr gui tools without too much pain... I had installed bzr via the .exe install that is completely uncompatible with the olive-gtk gui (!) I had to deinstall bzr and install the bzr python installer,(setup.py install as administrator). Now bzr viz and all other things works. Here is an interesting link: http://bazaar-vcs.org/MariusKruger#head-e234ddc518938afb140793d47320a2bee0b6 c8aa (a bit old, but you can easily find newer packages) and http://bazaar-vcs.org/Download (DO not use the standalone installer or will not have a working plugin directory!) Cheers, Laurent No virus found in this outgoing message. Checked by AVG. Version: 7.5.519 / Virus Database: 269.21.8/1340 - Release Date: 23/03/2008 18:50 From fperez.net at gmail.com Mon Mar 24 22:13:14 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 24 Mar 2008 19:13:14 -0700 Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk In-Reply-To: <47e835ea.0c07560a.5adf.ffffb9c5@mx.google.com> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> <47e835ea.0c07560a.5adf.ffffb9c5@mx.google.com> Message-ID: Hi Laurent, On Mon, Mar 24, 2008 at 4:14 PM, Laurent Dufr?chou wrote: > Hi all, > Where can I put some info on how to install bzr under Vista? > I've installed it with the .exe that is a big error and nobody must do that. > So I would like to warn newcomers how to set up bzr + bzr gui tools without > too much pain... That would be fantastic. Ideally, I'd like to have more of these docs in proper reST under source control, so I'd prefer it if you contribute it in the docs themselves. The right file for this would probably be http://ipython.scipy.org/doc/ipython1/html/development.html#version-control which is actually in this file (lovely bzr links, if anyone knows how to do better, let me know): http://bazaar.launchpad.net/~ipython/ipython/ipython1-dev/annotate/ondrej%40certik.cz-20080324124519-t9fsj9phqqzue1x1?file_id=development.txt-20080302203151-17poomviavsf2zjj-18 Eventually we should have a way to have users easily edit via the wiki these files, so that any user can contribute easily to the rest sources without even having to send a patch, just edit a wiki page and we get a patch automatically. Emanuelle (Gael's super-hacker girlfriend, who also participatedin the sprint!) has written some tools to start building some such functionality, and Stefan may push that further for numpy later, so I'm sure eventually we'll get all of that. In the meantime, just edit the source documents providing this information in the proper place. If you have any doubts and would like a review, feel free to work on it on a private branch on your +junk/ launchpad scratch area, so that others can look at it and help you out. Make a branch where these are the *only* changes so it's easy for someone else to pull them, review then and give you quick feedback without any danger of pulling changes to anything else. Also keep each commit reasonably small, so it's easy later to cherrypick if needed. We currently are nowhere near the necessary manpower to have a formal policy of patch review before commits (such as Sage uses, for example), but I think it's a good idea to get a bit of review when you're becoming familiar with new parts of the tree you hadn't gone into before. We're also trying to be much more strict with documentation thoroughness and testing in ipython1 than we ever were in ip0. Cheers, f From vivainio at gmail.com Tue Mar 25 04:17:33 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Tue, 25 Mar 2008 10:17:33 +0200 Subject: [IPython-dev] Docs for ipython1 In-Reply-To: References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> Message-ID: <46cb515a0803250117i48b3212dr9c334598a3dc2a7f@mail.gmail.com> On Tue, Mar 25, 2008 at 12:58 AM, Fernando Perez wrote: > And don't worry about ip0 for now, if you can help us later that's > great, but you've already done a lot! I'm doing the ip0 thing. There is already ipython.rst in doc/, but I need to fix the sphinx setup. It already creates the full document, but there are some warnings to clear up. We should have a pretty IPython0 document up soon, and I'll also make it a bit more up to date before 0.8.3. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Tue Mar 25 08:10:45 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 25 Mar 2008 05:10:45 -0700 Subject: [IPython-dev] Docs for ipython1 In-Reply-To: <46cb515a0803250117i48b3212dr9c334598a3dc2a7f@mail.gmail.com> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> <46cb515a0803250117i48b3212dr9c334598a3dc2a7f@mail.gmail.com> Message-ID: On Tue, Mar 25, 2008 at 1:17 AM, Ville M. Vainio wrote: > On Tue, Mar 25, 2008 at 12:58 AM, Fernando Perez wrote: > > > And don't worry about ip0 for now, if you can help us later that's > > great, but you've already done a lot! > > I'm doing the ip0 thing. There is already ipython.rst in doc/, but I > need to fix the sphinx setup. It already creates the full document, > but there are some warnings to clear up. > > We should have a pretty IPython0 document up soon, and I'll also make > it a bit more up to date before 0.8.3. Great! Cheers, f From ondrej at certik.cz Tue Mar 25 09:48:16 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Tue, 25 Mar 2008 14:48:16 +0100 Subject: [IPython-dev] IPython winpdb interface In-Reply-To: <47e7da01.0407560a.53ef.0285@mx.google.com> References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com> <47E6E860.30702@winpdb.org> <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com> <47e7da01.0407560a.53ef.0285@mx.google.com> Message-ID: <85b5c3130803250648u47b0e3a6qeceb76417c32b48e@mail.gmail.com> > > -----Message d'origine----- > De : ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De > la part de Ville M. Vainio > Envoy? : lundi 24 mars 2008 10:15 > ? : Nir; ipython-dev Mailing list > Objet : Re: [IPython-dev] IPython winpdb interface > > > > [CC-ing to ipython-dev, about adding winpdb support to IPython] > > On Mon, Mar 24, 2008 at 1:31 AM, Nir wrote: > > > Lets start with solving your initial scenario. > > Can you describe how is such an ipython session supposed to look like? > > If you now execute > > %run -d test.py > > IPython now sets the breakpoint at start of test.py and launches it in > pdb, running the test.py in ipython process. > > I'd like to add a new magic command %wpdb (in an extension, say > ipy_winpdb), so you can do > > %wpdb test.py > > which will do the same thing, i.e. launch test.py in the ipython > process, with rpdb2 breakpoint at start of the script. The rpdb2 > should only be started once in the IPython process. > > Likewise, if the ipy_winpdb extension is loaded and post-mortem > debugging is enabled, hitting an exception would break rpdb2 at the > exception. > > All of this is pretty easy in ipython side, we only need API support > in rpdb2 module to create a breakpoint at first valid position in a > file on file system. That's a very good idea indeed. I am also using winpdb almost exclusively for any debugging. Ondrej From vivainio at gmail.com Tue Mar 25 11:35:35 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Tue, 25 Mar 2008 17:35:35 +0200 Subject: [IPython-dev] IPython winpdb interface In-Reply-To: <47e7da01.0407560a.53ef.0285@mx.google.com> References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com> <47E6E860.30702@winpdb.org> <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com> <47e7da01.0407560a.53ef.0285@mx.google.com> Message-ID: <46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com> On Mon, Mar 24, 2008 at 6:42 PM, Laurent Dufr?chou wrote: > Hi ville, > > That's a GOOD idea :) I think it will be handy too (pdb is a bit dated, usagewise). Now it's up to Nir to create some kind of rpdb2 api we can use. > Does the %wpdb will: > - launch the winpdb GUI or > - only the server and we will have to launch the winpdb gui and attach to > the server manually or > - usage is only via rdpb server without winpdb GUI? We will only launch the rpdb2 server and let the user launch winpdb manually. The idea is to have rpdb2 running inside IPython process constantly, taking care of all the debugging needs... we can store the password to _ip.db, so it does not need to be entered every time IPython is launched. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From laurent.dufrechou at gmail.com Tue Mar 25 11:57:36 2008 From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=) Date: Tue, 25 Mar 2008 16:57:36 +0100 Subject: [IPython-dev] IPython winpdb interface In-Reply-To: <46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com> References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com> <47E6E860.30702@winpdb.org> <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com> <47e7da01.0407560a.53ef.0285@mx.google.com> <46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com> Message-ID: <47e920fb.0405560a.1927.ffffd4d4@mx.google.com> Ok I see, perhaps Nir will confirm, but if you manage to make rpdb server run inside ipython, I think we will be able to launch the GUI and set breakpoint and do step in/out etc...(this is what you say no?) And I'm quite sure that it ever exist an API to manage it (if you want to control it via ipython) because Nir said me a long time ago that PIDA was using it like this...(if I remember well)... In which bzr branch do you plan to play with this, mtexp or stable one?? -----Message d'origine----- De?: Ville M. Vainio [mailto:vivainio at gmail.com] Envoy??: mardi 25 mars 2008 16:36 ??: Laurent Dufr?chou Cc?: Nir; ipython-dev Mailing list Objet?: Re: [IPython-dev] IPython winpdb interface On Mon, Mar 24, 2008 at 6:42 PM, Laurent Dufr?chou wrote: > Hi ville, > > That's a GOOD idea :) I think it will be handy too (pdb is a bit dated, usagewise). Now it's up to Nir to create some kind of rpdb2 api we can use. > Does the %wpdb will: > - launch the winpdb GUI or > - only the server and we will have to launch the winpdb gui and attach to > the server manually or > - usage is only via rdpb server without winpdb GUI? We will only launch the rpdb2 server and let the user launch winpdb manually. The idea is to have rpdb2 running inside IPython process constantly, taking care of all the debugging needs... we can store the password to _ip.db, so it does not need to be entered every time IPython is launched. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Tue Mar 25 12:38:23 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Tue, 25 Mar 2008 18:38:23 +0200 Subject: [IPython-dev] IPython winpdb interface In-Reply-To: <47e920fb.0405560a.1927.ffffd4d4@mx.google.com> References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com> <47E6E860.30702@winpdb.org> <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com> <47e7da01.0407560a.53ef.0285@mx.google.com> <46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com> <47e920fb.0405560a.1927.ffffd4d4@mx.google.com> Message-ID: <46cb515a0803250938h5ecfc531hb8933bf18a4b579b@mail.gmail.com> On Tue, Mar 25, 2008 at 5:57 PM, Laurent Dufr?chou wrote: > And I'm quite sure that it ever exist an API to manage it (if you want to > control it via ipython) because Nir said me a long time ago that PIDA was > using it like this...(if I remember well)... If I understood correctly. no such API exist yet. > In which bzr branch do you plan to play with this, mtexp or stable one?? Depends on time we get an API. This will end up post-0.8.3. BTW, mtexp is dead, it has already been integrated to trunk. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Tue Mar 25 15:42:11 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Tue, 25 Mar 2008 21:42:11 +0200 Subject: [IPython-dev] IPython winpdb interface In-Reply-To: <47E939BA.4080104@winpdb.org> References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com> <47E6E860.30702@winpdb.org> <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com> <47e7da01.0407560a.53ef.0285@mx.google.com> <46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com> <47E939BA.4080104@winpdb.org> Message-ID: <46cb515a0803251242g44663a5ch35bdaaa3c9193799@mail.gmail.com> On Tue, Mar 25, 2008 at 7:43 PM, Nir wrote: > To do that type in ipython: > import rpdb2; rpdb2.start_embedded_debugger() Yes, that is not a problem. > If I understand Ville, he is interested in usability similar to ipydb. > This can be arranged. I am interested in setting breakpoints programmatically - this is needed for %run -d like functionality. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From bialix at ukr.net Tue Mar 25 16:07:47 2008 From: bialix at ukr.net (Alexander Belchenko) Date: Tue, 25 Mar 2008 22:07:47 +0200 Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk In-Reply-To: <47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> <47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com> Message-ID: <47E95B93.2050004@ukr.net> Laurent Dufr?chou ?????: > Hi all, > Where can I put some info on how to install bzr under Vista? It will be nice if you put the copy of your instruction to http://bazaar-vcs.org. It's a wiki in the end. There is already instructions how to install it: http://bazaar-vcs.org/WindowsInstall Can you tell me is this instructions is wrong or not good enough? > I've installed it with the .exe that is a big error and nobody must do that. It's shame for me personally to hear this. > So I would like to warn newcomers how to set up bzr + bzr gui tools without > too much pain... QBzr GUI is compatible with bzr.exe. Of course, it's not the GTK one. > I had installed bzr via the .exe install that is completely uncompatible > with the olive-gtk gui (!) IMO, it's not true. Here is package of bzr-gtk compatible with bzr.exe: http://d5190871.u44.websitesource.net/bzr/ There is bzr-gtk-0.93.0 release only, but I don't follow bzr-gtk development, so I'm not sure what's latest release is. > I had to deinstall bzr and install the bzr python installer,(setup.py > install as administrator). > Now bzr viz and all other things works. > > Here is an interesting link: > http://bazaar-vcs.org/MariusKruger#head-e234ddc518938afb140793d47320a2bee0b6 > c8aa > (a bit old, but you can easily find newer packages) > > and > http://bazaar-vcs.org/Download > (DO not use the standalone installer or will not have a working plugin > directory!) I don't understand this warning. Can you explain it a bit more? From vivainio at gmail.com Tue Mar 25 16:06:16 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Tue, 25 Mar 2008 22:06:16 +0200 Subject: [IPython-dev] IPython winpdb interface In-Reply-To: <47E9596F.7030304@winpdb.org> References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com> <47E6E860.30702@winpdb.org> <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com> <47e7da01.0407560a.53ef.0285@mx.google.com> <46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com> <47E939BA.4080104@winpdb.org> <46cb515a0803251242g44663a5ch35bdaaa3c9193799@mail.gmail.com> <47E9596F.7030304@winpdb.org> Message-ID: <46cb515a0803251306m5fcb412amfcf693584f893a46@mail.gmail.com> On Tue, Mar 25, 2008 at 9:58 PM, Nir wrote: > > I am interested in setting breakpoints programmatically - this is > > needed for %run -d like functionality. > > > Do you intend to have a debugger console? Not really. Being able to set the initial brekapoint is enough. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From laurent.dufrechou at gmail.com Tue Mar 25 17:10:34 2008 From: laurent.dufrechou at gmail.com (=?UTF-8?Q?Laurent_Dufr=C3=A9chou?=) Date: Tue, 25 Mar 2008 22:10:34 +0100 Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk In-Reply-To: <47E95B93.2050004@ukr.net> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> <47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com> <47E95B93.2050004@ukr.net> Message-ID: <47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com> Hi Alexander, You are the maintainer of the .exe version?? In fact I've felt in some problem installing the .exe. Sorry for my previous email, it was more a warning on my usage case than a 'these guys have done a bad job'! So let me explain my problems (and perhaps you will tell that I've done somethng bad!), if not let's find solutions to improve things :) I've used bazar first installing bazar with the .exe install. (1.3 version) It added bzr here in the context menu when you right click on a directory. Since the install the context seemed sluggish, AND bugguy. (perhaps it was a problem with my vista but uninstalling it resolved the problem.) (I can retry if you want just to chek this) By bugguy I mean, that selecting an option in right click context menu made the action and made the menu appears an other time (!) The instruction was good but: GTK+ Runtime link is obsolete.(will modify it with one I found, it was really hard to found, well I was bad on this point...). http://sourceforge.net/project/showfiles.php?group_id=71914 Bazaar Gui didn't worked at all with the bzr 1.3.0 .exe install. What I saw was: It created a directory in c:/program files/bazaar/ And the bazaar GUI install itself in c:/python25/lib/site-package/bazaar/plugins But nothing else was in this directory the site-package/bazar one. So the olive gtk didn't work at all. Using http://bazaar-vcs.org/MariusKruger#head-e234ddc518938afb140793d47320a2bee0b6c8aa I uninstalled the .exe version and installed a recent python install version (setup install.py) and then olive-gtk worked. The .Bat file works fine. I've done a .ico if you want it to add it somewhere to have a beautifull shortcut. How and last problem I encoutered, don't know why bit I had to add: libglade-2.0-0.dll libxml2.dll rename zlib1.dll into zlib.dll in C:\Program Files\GTK2-Runtime\lib To make olive-gtk work. I must have missed something there... On your reply: > http://d5190871.u44.websitesource.net/bzr/ Ok I've not tried this olive-gtk.exe perhaps mixing bzr.exe with olive-gtk.gz install was a bad idea, but the wiki don't warn about this. > It's shame for me personally to hear this. Sorry. >QBzr GUI is compatible with bzr.exe. Of course, it's not the GTK one. Ok will try this! Regards, Laurent -----Message d'origine----- De : Alexander Belchenko [mailto:bialix at ukr.net] Envoy? : mardi 25 mars 2008 21:08 ? : Laurent Dufr?chou Cc : 'ipython user list'; 'ipython-dev Mailing list' Objet : Re: Bzr install under vista + bzr olive-gtk Laurent Dufr?chou ?????: > Hi all, > Where can I put some info on how to install bzr under Vista? It will be nice if you put the copy of your instruction to http://bazaar-vcs.org. It's a wiki in the end. There is already instructions how to install it: http://bazaar-vcs.org/WindowsInstall Can you tell me is this instructions is wrong or not good enough? > I've installed it with the .exe that is a big error and nobody must do that. It's shame for me personally to hear this. > So I would like to warn newcomers how to set up bzr + bzr gui tools without > too much pain... QBzr GUI is compatible with bzr.exe. Of course, it's not the GTK one. > I had installed bzr via the .exe install that is completely uncompatible > with the olive-gtk gui (!) IMO, it's not true. Here is package of bzr-gtk compatible with bzr.exe: http://d5190871.u44.websitesource.net/bzr/ There is bzr-gtk-0.93.0 release only, but I don't follow bzr-gtk development, so I'm not sure what's latest release is. > I had to deinstall bzr and install the bzr python installer,(setup.py > install as administrator). > Now bzr viz and all other things works. > > Here is an interesting link: > http://bazaar-vcs.org/MariusKruger#head-e234ddc518938afb140793d47320a2bee0b6 > c8aa > (a bit old, but you can easily find newer packages) > > and > http://bazaar-vcs.org/Download > (DO not use the standalone installer or will not have a working plugin > directory!) I don't understand this warning. Can you explain it a bit more? From bialix at ukr.net Tue Mar 25 18:23:27 2008 From: bialix at ukr.net (Alexander Belchenko) Date: Wed, 26 Mar 2008 00:23:27 +0200 Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk In-Reply-To: <47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> <47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com> <47E95B93.2050004@ukr.net> <47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com> Message-ID: <47E97B5F.4090901@ukr.net> Laurent Dufr?chou ?????: > Hi Alexander, Bonjour/Bonsoir, > You are the maintainer of the .exe version?? I build all 3 installers for windows. > In fact I've felt in some problem installing the .exe. > Sorry for my previous email, it was more a warning on my usage case than a 'these guys have done a bad job'! OK. > So let me explain my problems (and perhaps you will tell that I've done somethng bad!), if not let's find solutions to improve things :) > > I've used bazar first installing bazar with the .exe install. (1.3 version) > It added bzr here in the context menu when you right click on a directory. > Since the install the context seemed sluggish, AND bugguy. Thank you to remind me about this misfeature. I made context menu after looking at cygwin's implementation but did something wrong, and in all recent releases it disabled by default. I'm leaning to remove it at all in next release. > (perhaps it was a problem with my vista but uninstalling it resolved the problem.) > (I can retry if you want just to chek this) By bugguy I mean, that selecting an option in right click context menu made the action and made the menu appears an other time (!) > > The instruction was good but: > GTK+ Runtime link is obsolete.(will modify it with one I found, it was really hard to found, well I was bad on this point...). > http://sourceforge.net/project/showfiles.php?group_id=71914 Sorry, I'm not the author of the page http://bazaar-vcs.org/WindowsInstall. You need only register on bazaar-vcs.org to get access to editing this page. Feel free to fix broken links. > Bazaar Gui didn't worked at all with the bzr 1.3.0 .exe install. There is no 'Bazaar Gui'. There is bzr-gtk plugin, that is not the part of bzr core, and therefore is not the part of standalone bzr.exe. Plain bzr-gtk is not working because bzr.exe does not contains inside any required PyGTK libraries. > What I saw was: > It created a directory in c:/program files/bazaar/ > And the bazaar GUI install itself in c:/python25/lib/site-package/bazaar/plugins You're using wrong installer of bzr-gtk. You need installer especially created to work with bzr.exe. Such installer should have PyGTK/GTK libs inside as well. I'm not maintainer of installer for bzr-gtk. Because I use QBzr. > But nothing else was in this directory the site-package/bazar one. > So the olive gtk didn't work at all. > Using http://bazaar-vcs.org/MariusKruger#head-e234ddc518938afb140793d47320a2bee0b6c8aa I uninstalled the .exe version and installed a recent python install version (setup install.py) and then olive-gtk worked. > > The .Bat file works fine. I've done a .ico if you want it to add it somewhere to have a beautifull shortcut. > > How and last problem I encoutered, don't know why bit I had to add: > libglade-2.0-0.dll > libxml2.dll > rename zlib1.dll into zlib.dll > in C:\Program Files\GTK2-Runtime\lib > To make olive-gtk work. > I must have missed something there... Sorry, IMHO GTK toolkit is too alien in Windows environment. From all GTK universe I'm using GIMP, but only because I'm too lazy to look for better free non-GTK tool. > > On your reply: >> http://d5190871.u44.websitesource.net/bzr/ > Ok I've not tried this olive-gtk.exe perhaps mixing bzr.exe with olive-gtk.gz install was a bad idea, but the wiki don't warn about this. I mean this one: http://d5190871.u44.websitesource.net/bzr/bzr-gtk-0.93.0-setup-1.exe From description and its content it seems like it's designed to work with bzr.exe, but I'm not sure. At least it should install 'gtk' plugin to C:\Program Files\Bazaar\plugins\ > >> QBzr GUI is compatible with bzr.exe. Of course, it's not the GTK one. > Ok will try this! New version will be released soon: http://permalink.gmane.org/gmane.comp.version-control.bazaar-ng.general/39104 From laurent.dufrechou at gmail.com Tue Mar 25 18:55:46 2008 From: laurent.dufrechou at gmail.com (=?UTF-8?Q?Laurent_Dufr=C3=A9chou?=) Date: Tue, 25 Mar 2008 23:55:46 +0100 Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk In-Reply-To: <47E97B5F.4090901@ukr.net> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> <47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com> <47E95B93.2050004@ukr.net> <47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com> <47E97B5F.4090901@ukr.net> Message-ID: <47e98300.0aa1660a.2a5a.2731@mx.google.com> Ok alanxander, i've updated the wiki. I've also installed QBzr 0.8: https://launchpad.net/qbzr/trunk/0.8.0/+download/qbzr-0.8.0.win32.exe Files have been installed in bzr/plugins but there is no script to launch.(Qbzr or something like this?) Do I Miss something? From fperez.net at gmail.com Tue Mar 25 19:19:07 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 25 Mar 2008 16:19:07 -0700 Subject: [IPython-dev] IPython winpdb interface In-Reply-To: <46cb515a0803250938h5ecfc531hb8933bf18a4b579b@mail.gmail.com> References: <46cb515a0803231331k2330817cp4ef9cb22c3ebf5db@mail.gmail.com> <47E6E860.30702@winpdb.org> <46cb515a0803240214h61e857fdl35317b4204add12c@mail.gmail.com> <47e7da01.0407560a.53ef.0285@mx.google.com> <46cb515a0803250835g7bf64471p80bda793559c0cf8@mail.gmail.com> <47e920fb.0405560a.1927.ffffd4d4@mx.google.com> <46cb515a0803250938h5ecfc531hb8933bf18a4b579b@mail.gmail.com> Message-ID: On Tue, Mar 25, 2008 at 9:38 AM, Ville M. Vainio wrote: > > In which bzr branch do you plan to play with this, mtexp or stable one?? > > Depends on time we get an API. This will end up post-0.8.3. > > BTW, mtexp is dead, it has already been integrated to trunk. Then it should be removed from launchpad, to reduce confusion. Let's keep a habit of killing branches as soon as their intended purpose is finished. Also, let's try to use all the labelling features of launchpad to properly indicate what a branch is meant for. Here's an example of what I mean (I used both the title and the details field to clarify the intent of that branch to potential collaborators): https://code.launchpad.net/~nipy-developers/nipy/fff2 Cheers, f From ondrej at certik.cz Tue Mar 25 20:19:02 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Wed, 26 Mar 2008 01:19:02 +0100 Subject: [IPython-dev] mercurial now has free hosting too Message-ID: <85b5c3130803251719pde17151x96761bb8cd2b4f7e@mail.gmail.com> Hi, since there was so much discussion whether bzr or hg, Mercurial has now free hosting too: http://freehg.org/ also Mercurial 1.0 was finally released yesterday. Bzr has Launchpad, that's one of the (main) reasons ipython is investigating it, so I am still learning how to use bzr, but it's not really so much different. The only little annoying thing I discovered so far is that it feels slower than hg, on regular tasks, like "bzr st", "bzr pull", "bzr up", etc. Only a little bit, but still it creates the feeling in me, that something is missing -- it's like if you are used to a fast car and then you get into a slower car - even though it's fast enough to drive you to the shop, you still are missing something, at least I am. :) But generally I wanted to say, that I think bzr is a good choice too. Ondrej From vivainio at gmail.com Wed Mar 26 05:36:18 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 26 Mar 2008 11:36:18 +0200 Subject: [IPython-dev] mercurial now has free hosting too In-Reply-To: <85b5c3130803251719pde17151x96761bb8cd2b4f7e@mail.gmail.com> References: <85b5c3130803251719pde17151x96761bb8cd2b4f7e@mail.gmail.com> Message-ID: <46cb515a0803260236r2a4685eeg7f2bacda6d043f81@mail.gmail.com> On Wed, Mar 26, 2008 at 2:19 AM, Ondrej Certik wrote: > since there was so much discussion whether bzr or hg, Mercurial has > now free hosting too: I think the killer issue here is Launchpad. We get pretty much everything for free, and it's something that we can expect to stay around in the long haul (and will continue to improve). The slight performance disadvantage of bzr is a minor concern, esp. for the project with the size of IPython. Both hg and bzr as-such (if we forget the hosting options etc) are "good enough" right now, and both are being improved. As it stands, I think we should stick with LP + bzr for now - we can re-evaluate the situation a year or so down the line, if need be. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From gael.varoquaux at normalesup.org Wed Mar 26 05:40:47 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Wed, 26 Mar 2008 10:40:47 +0100 Subject: [IPython-dev] mercurial now has free hosting too In-Reply-To: <46cb515a0803260236r2a4685eeg7f2bacda6d043f81@mail.gmail.com> References: <85b5c3130803251719pde17151x96761bb8cd2b4f7e@mail.gmail.com> <46cb515a0803260236r2a4685eeg7f2bacda6d043f81@mail.gmail.com> Message-ID: <20080326094047.GB7186@phare.normalesup.org> On Wed, Mar 26, 2008 at 11:36:18AM +0200, Ville M. Vainio wrote: > I think the killer issue here is Launchpad. We get pretty much > everything for free, and it's something that we can expect to stay > around in the long haul (and will continue to improve). The slight > performance disadvantage of bzr is a minor concern, esp. for the > project with the size of IPython. > Both hg and bzr as-such (if we forget the hosting options etc) are > "good enough" right now, and both are being improved. As it stands, I > think we should stick with LP + bzr for now - we can re-evaluate the > situation a year or so down the line, if need be. I have the very exact same gut fealing than you. I have watch developpers switch to DVCS lately, and I must say it requires a certain change in working habits. Let us wait for people to get used to these new tools before discussing which one to you. And don't get me wrong, I love DVCS, I just don't want to lose developper because of it. Ga?l From jorgen.stenarson at bostream.nu Wed Mar 26 14:09:26 2008 From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=) Date: Wed, 26 Mar 2008 19:09:26 +0100 Subject: [IPython-dev] pyreadline Message-ID: <47EA9156.2050603@bostream.nu> Hi all, I have been thinking about the future of pyreadline considering the work on ipython1. * ANSI color escapes. In trunk ansi color escapes are used to handle color in prompts. This is convenient(for the programmer) when the terminal understands these escapes. But on windows this is not the case, I imagine a few of the guis proposed for ipython1 would also have trouble with them. In pyreadline we have code for parsing this. So it would probably make sense to have this functionality available for ipython1 as well. From a users point of view I think we should consider figuring out a way that is easier to understand and expand for the future. We could include ways to use boldface, italics, underline etc. * I know Gael is looking into using pyreadline for some gui interface this means there will be even more dependency of ipython on pyreadline. Perhaps this even means we should add pyreadline to ipython1. * Completions. Parts of the tab-completion code in ipython could probably be made to work in pyreadline directly making it useful at a normal pythonprompt as well. There is a ton of work I would like to do in cleaning up the codebase and improving the test coverage. Going with nose here just as in ipython makes sense. Converting the documentation to rest format using sphinx looks like a good idea as well. J?rgen From laurent.dufrechou at gmail.com Wed Mar 26 16:47:27 2008 From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=) Date: Wed, 26 Mar 2008 21:47:27 +0100 Subject: [IPython-dev] pyreadline In-Reply-To: <47EA9156.2050603@bostream.nu> References: <47EA9156.2050603@bostream.nu> Message-ID: <47eab66f.25e2660a.156e.0565@mx.google.com> Hi jorg?n, Ga?l explained me a little what was a readline. Now I better understand how it interface with a gui. (Please any GUI programmer correct me if I'm wrong, especially for cocoa or QT) * ANSI color escapes: In a gui we use a widget that has methods to color the text it display, depending on the widget used, the methods are way different. When I interfaced wx to ipython, I sended a string to ipython shell and it sent back (via readline interface?) an ansi color escaped string. I had to parse it and process it to make it appears the way it was expected. >From my point of view I don't expect readline to manage to handle my widget display. I mean, send me the data the way I can process it and I will be happy. Readline must be (I think) GUI agnostic. Now we can discuss how you can communicate the data. It would be really cool to be able to select an output object. For example, we could select to set readline output as escape character OR for example (if we want to handle bolt etc easily) as HTML output.Why html? Because some widget on WX can get HTML and display it as is. We can imagine also XML output or whatelse a GUI guy would need. *Completion: If ipython1 manage it by itself, I'm not sure readline should manage it too. Or does it provide more functions? *History >From what I understood from Gael, readline is the object that manage history. What I needed when I wrote Wx shell for ip0, was: getHistoryBack(),getHistoryWidth(),getHistory() etc... in fact function that can provide me a way to get and manage history strings. So if we could define a simple interface to readline that could be good. One important point, gael tells me that usually readline get all character received from a frontend and it handle it. I think, (perhaps I'm wrong) that from a GUI designer point of view, I would greatly prefer to send to readline full string instead of character. Key event handling is tighly managed by gui itself, so I think it is a little complex to send it to readline with readline being GUI agnostic... Well I think we will have to discuss this point. QT/Cocoa guys, what do you think about this? Perhaps the solution is to create a wxreadline that will be a thin wrapper between wx key event and readline processing? Don't have a real solution there... Be careful, I was speaking from a GUI designer point of view, from a shell frontend point of view I understand the way that readline process char by char the data... So that why, if you explain the advantage of calling readline each time a char is processed, I will be willing to try to use it like a shell frontend designer would.(to have a unified interface) Laurent -----Message d'origine----- De?: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De la part de J?rgen Stenarson Envoy??: mercredi 26 mars 2008 19:09 ??: IPython-dev List Objet?: [IPython-dev] pyreadline Hi all, I have been thinking about the future of pyreadline considering the work on ipython1. * ANSI color escapes. In trunk ansi color escapes are used to handle color in prompts. This is convenient(for the programmer) when the terminal understands these escapes. But on windows this is not the case, I imagine a few of the guis proposed for ipython1 would also have trouble with them. In pyreadline we have code for parsing this. So it would probably make sense to have this functionality available for ipython1 as well. From a users point of view I think we should consider figuring out a way that is easier to understand and expand for the future. We could include ways to use boldface, italics, underline etc. * I know Gael is looking into using pyreadline for some gui interface this means there will be even more dependency of ipython on pyreadline. Perhaps this even means we should add pyreadline to ipython1. * Completions. Parts of the tab-completion code in ipython could probably be made to work in pyreadline directly making it useful at a normal pythonprompt as well. There is a ton of work I would like to do in cleaning up the codebase and improving the test coverage. Going with nose here just as in ipython makes sense. Converting the documentation to rest format using sphinx looks like a good idea as well. J?rgen _______________________________________________ IPython-dev mailing list IPython-dev at scipy.org http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev From gael.varoquaux at normalesup.org Wed Mar 26 16:56:45 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Wed, 26 Mar 2008 21:56:45 +0100 Subject: [IPython-dev] pyreadline In-Reply-To: <47eab66f.25e2660a.156e.0565@mx.google.com> References: <47EA9156.2050603@bostream.nu> <47eab66f.25e2660a.156e.0565@mx.google.com> Message-ID: <20080326205645.GC24201@phare.normalesup.org> On Wed, Mar 26, 2008 at 09:47:27PM +0100, Laurent Dufr?chou wrote: > One important point, gael tells me that usually readline get all > character > received from a frontend and it handle it. > I think, (perhaps I'm wrong) that from a GUI designer point of view, I > would > greatly prefer to send to readline full string instead of character. > Key > event handling is tighly managed by gui itself, so I think it is a > little > complex to send it to readline with readline being GUI agnostic... OK, but how to you choose how a key must behave? You need one single object that knows the configuration of keys (is pageup history, or nothing?), and what if I want to switch to VI mode, you don't want to implement this in all the frontends. Sending a string defeats the goal of readline: readline is what translates keystrokes to a command string. Readline knows that if I use "ctr-left", it should jump the cursor one word to the left. Yes, it is not easy, but that's what readline is about, IMHO. Ga?l From laurent.dufrechou at gmail.com Wed Mar 26 20:34:39 2008 From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=) Date: Thu, 27 Mar 2008 01:34:39 +0100 Subject: [IPython-dev] pyreadline (Hey cocoa/QT/wx ipython GUI what do you think about this?) In-Reply-To: <20080326205645.GC24201@phare.normalesup.org> References: <47EA9156.2050603@bostream.nu> <47eab66f.25e2660a.156e.0565@mx.google.com> <20080326205645.GC24201@phare.normalesup.org> Message-ID: <47eaebb1.0c58560a.2b21.16d9@mx.google.com> Hi ga?l, >OK, but how to you choose how a key must behave? You need one single >object that knows the configuration of keys (is pageup history, or >nothing?), and what if I want to switch to VI mode, you don't want to >implement this in all the frontends. When you use a widget, like scintilla one that I used, all the classical key where already handled by the text widget itself. I add to overload some to better match ipython shell behavior. Like pushing 'up', does not go up but call history back. So if you want VI mode , just design your ipython widget around a VI aware text widget, that's all... >Sending a string defeats the goal of readline: readline is what >translates keystrokes to a command string. Readline knows that if I use >"ctr-left", it should jump the cursor one word to the left. Yes, it is >not easy, but that's what readline is about, IMHO. Ok I've deleted the beginning of my email: In fact the problem can be solved 'easily'. For WX the widget developer will ahev something like this: Class MyTextAreaWidget(object): def __init__(self): ... Self.readline = Readline(ipython_instance) ... self.readline.MoveCursorLeft = self.MoveCursorLeft self.readline.GotoStartOfLine = self.MoveToPrompt #------------------------ Key Handler ------------------------------------ def keyPress(self, event): ''' Key press callback with plenty of shell goodness, like history, autocompletions, etc. ''' #readline shortcircuit if event.GetKeyCode() == ord('C'): if event.Modifiers == wx.MOD_CONTROL: #we raise an exception inside the IPython thread container self.IP.ce.raise_exc(KeyboardInterrupt) return #readline key handling self.readline.ProcessKey(event.GetKeycode()) #post key processing if (self.readline.has_processed_key == False): self.postProcessKey() Developer must have the possibility to let or not readline to handle the keys. If he don't want to use readline key handling, he must have access to function that can make ipython execute a string, and that should be ok. Am I clear in my explanation? Readline only process key but does not manage widget directly. User will to hook it's widget function to readline one. With this approach I'm OK. So we will be able to use a text area renderer, a scintilla renderer or even an html renderer... This will not be a problem for readline, has it will only redirect key event to good widget functions via a unified interface. Laurent From gael.varoquaux at normalesup.org Thu Mar 27 04:55:10 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Thu, 27 Mar 2008 09:55:10 +0100 Subject: [IPython-dev] pyreadline (Hey cocoa/QT/wx ipython GUI what do you think about this?) In-Reply-To: <47eaebb1.0c58560a.2b21.16d9@mx.google.com> References: <47EA9156.2050603@bostream.nu> <47eab66f.25e2660a.156e.0565@mx.google.com> <20080326205645.GC24201@phare.normalesup.org> <47eaebb1.0c58560a.2b21.16d9@mx.google.com> Message-ID: <20080327085510.GB3109@phare.normalesup.org> On Thu, Mar 27, 2008 at 01:34:39AM +0100, Laurent Dufr?chou wrote: > When you use a widget, like scintilla one that I used, all the > classical key > where already handled by the text widget itself. I add to overload some > to > better match ipython shell behavior. Like pushing 'up', does not go up > but > call history back. > So if you want VI mode , just design your ipython widget around a VI > aware > text widget, that's all... If you head in this direction you are going to repeat the functionnality in each front end. As a result you will have more code to maintain, and inconsistent behavior between widgets. > Developer must have the possibility to let or not readline to handle > the > keys. > If he don't want to use readline key handling, he must have access to > function that can make ipython execute a string, and that should be ok. I just don't see why. The fonction call overhead is not the end of the world. It is true that finding a decent interface between the readline and the front end is hard, but it will make future development easier, and the code easier to read. Anyhow, this is all theory for now, and their is a lot of work before we actually face these problems. > Readline only process key but does not manage widget directly. Well, it gives back high-level instructions on the widget describing how it behaves. Finding this set of high-level instructions is the key to having a good decoupling between front ends and ipython. Ga?l From walter at livinglogic.de Thu Mar 27 04:57:42 2008 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu, 27 Mar 2008 09:57:42 +0100 Subject: [IPython-dev] pyreadline In-Reply-To: <47EA9156.2050603@bostream.nu> References: <47EA9156.2050603@bostream.nu> Message-ID: <47EB6186.7070308@livinglogic.de> J?rgen Stenarson wrote: > Hi all, > > I have been thinking about the future of pyreadline considering the work > on ipython1. > > * ANSI color escapes. In trunk ansi color escapes are used to handle > color in prompts. This is convenient(for the programmer) when the > terminal understands these escapes. But on windows this is not the case, > I imagine a few of the guis proposed for ipython1 would also have > trouble with them. In pyreadline we have code for parsing this. So it > would probably make sense to have this functionality available for > ipython1 as well. From a users point of view I think we should consider > figuring out a way that is easier to understand and expand for the > future. We could include ways to use boldface, italics, underline etc. What I did for the ipipe stuff is that the output for an object comes from a generator that looks like this: def repr_list(obj): yield (normal, '[') for (i, child) in enumerate(obj): if i: yield (normal, ', ') for (color, part) in specialrepr(child): yield (color, part) yield (normal, '[') i.e. the generator produces tuples with a string and a color object for that string. The advantage of a generator is that you can cancel the generator once the screen is filled, so you don't have to produce output that you are not displaying anyway. Currently in ipipe this color object is a wrapper for curses/ANSI colors, i.e. it knows about 8 foreground colors, 8 background colors, bold and underline. I have a half finished version lying around, where forground and background color are three floats (i.e. Color(0., 0., 0.) is black, Color(1., 1., 1.) is white). For terminal and curses output this gets mapped to the nearest of the 16 ANSI colors (it also supports 256 color terminals). For this to be usable we probably would need theme support, because for terminal output you might very well have a black background, for GUIs you almost never have. Servus, Walter From bialix at ukr.net Fri Mar 28 03:56:49 2008 From: bialix at ukr.net (Alexander Belchenko) Date: Fri, 28 Mar 2008 09:56:49 +0200 Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk In-Reply-To: <47e98300.0aa1660a.2a5a.2731@mx.google.com> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> <47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com> <47E95B93.2050004@ukr.net> <47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com> <47E97B5F.4090901@ukr.net> <47e98300.0aa1660a.2a5a.2731@mx.google.com> Message-ID: <47ECA4C1.9080203@ukr.net> Laurent Dufr?chou ?????: > Ok alanxander, i've updated the wiki. > I've also installed QBzr 0.8: > https://launchpad.net/qbzr/trunk/0.8.0/+download/qbzr-0.8.0.win32.exe > > Files have been installed in bzr/plugins but there is no script to launch.(Qbzr or something like this?) > Do I Miss something? QBzr is like bzr-gtk (not Olive): it provides a set of commands similar to basic bzr commands, but with GUI interface. QBzr 0.8 provides: qdiff, qlog, qcommit, qannotate, qbrowse, qconfig. You need to cd to your branch and launch them from command line. May be it's not very convenient if you're like Olive program, but currently there is no QBzr standalone program yet. From laurent.dufrechou at gmail.com Fri Mar 28 05:00:34 2008 From: laurent.dufrechou at gmail.com (=?utf-8?Q?Laurent_Dufr=C3=A9chou?=) Date: Fri, 28 Mar 2008 10:00:34 +0100 Subject: [IPython-dev] Bzr install under vista + bzr olive-gtk In-Reply-To: <47ECA4C1.9080203@ukr.net> References: <46cb515a0803240230g6c910c88kfb3e2c54008a322b@mail.gmail.com> <85b5c3130803240550o3b562473r8d1ec1e9f1cfb2aa@mail.gmail.com> <47e835ea.0c07560a.5adf.ffffb9c5__17398.729947354$1206400555$gmane$org@mx.google.com> <47E95B93.2050004@ukr.net> <47e96a57.0d135e0a.0da4.ffff8e16@mx.google.com> <47E97B5F.4090901@ukr.net> <47e98300.0aa1660a.2a5a.2731@mx.google.com> <47ECA4C1.9080203@ukr.net> Message-ID: <47ecb3cb.0437560a.7752.2810@mx.google.com> Ok, i found how to use it! bzr qannotate myfile.py etc... thx -----Message d'origine----- De : Alexander Belchenko [mailto:bialix at ukr.net] Envoy? : vendredi 28 mars 2008 08:57 ? : Laurent Dufr?chou Cc : 'ipython user list'; 'ipython-dev Mailing list' Objet : Re: Bzr install under vista + bzr olive-gtk Laurent Dufr?chou ?????: > Ok alanxander, i've updated the wiki. > I've also installed QBzr 0.8: > https://launchpad.net/qbzr/trunk/0.8.0/+download/qbzr-0.8.0.win32.exe > > Files have been installed in bzr/plugins but there is no script to launch.(Qbzr or something like this?) > Do I Miss something? QBzr is like bzr-gtk (not Olive): it provides a set of commands similar to basic bzr commands, but with GUI interface. QBzr 0.8 provides: qdiff, qlog, qcommit, qannotate, qbrowse, qconfig. You need to cd to your branch and launch them from command line. May be it's not very convenient if you're like Olive program, but currently there is no QBzr standalone program yet. From glenn at tarbox.org Fri Mar 28 15:29:30 2008 From: glenn at tarbox.org (Glenn H Tarbox, PhD) Date: Fri, 28 Mar 2008 12:29:30 -0700 Subject: [IPython-dev] Twisted 8... Message-ID: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org> Ipython1 no longer compiles with the latest Twisted trunk because of Twisted's new numbering scheme. Fails in the version check. I'm only aware of one significant IPython1 refactoring issue relating to the blockingThread capability now in the Twisted mainline. (probably not refactoring, just an adjustment to use Twisted's call rather than whatever version IPython1 was using in anticipation of this new twisted feature.) -glenn -- Glenn H. Tarbox, PhD | Don?t worry about people stealing your ideas. If your ideas 206-494-0819 | are any good, you?ll have to ram them down people?s throats glenn at tarbox.org | ?Howard Aiken, IBM engineer From ellisonbg.net at gmail.com Fri Mar 28 16:00:13 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 28 Mar 2008 14:00:13 -0600 Subject: [IPython-dev] Twisted 8... In-Reply-To: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org> References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org> Message-ID: <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> Glenn, I am plannig to get IPython1 working with Twisted 8.0 soon. As you mention, there should only be a few things we have to do. How do people feel about requiring the new version of Twisted? It is a significant release, and Twisted 2.5.0 had been out for a very long time. Brian On Fri, Mar 28, 2008 at 1:29 PM, Glenn H Tarbox, PhD wrote: > Ipython1 no longer compiles with the latest Twisted trunk because of > Twisted's new numbering scheme. Fails in the version check. > > I'm only aware of one significant IPython1 refactoring issue relating to > the blockingThread capability now in the Twisted mainline. (probably not > refactoring, just an adjustment to use Twisted's call rather than > whatever version IPython1 was using in anticipation of this new twisted > feature.) > > -glenn > > -- > Glenn H. Tarbox, PhD | Don't worry about people stealing your ideas. If your ideas > 206-494-0819 | are any good, you'll have to ram them down people's throats > glenn at tarbox.org | ?Howard Aiken, IBM engineer > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From steve at shrogers.com Fri Mar 28 17:51:11 2008 From: steve at shrogers.com (Steven H. Rogers) Date: Fri, 28 Mar 2008 15:51:11 -0600 (MDT) Subject: [IPython-dev] Twisted 8... In-Reply-To: <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org> <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> Message-ID: <11946.192.55.12.36.1206741071.squirrel@mail2.webfaction.com> On Fri, March 28, 2008 14:00, Brian Granger wrote: > Glenn, > > I am plannig to get IPython1 working with Twisted 8.0 soon. As you > mention, there should only be a few things we have to do. How do > people feel about requiring the new version of Twisted? It is a > significant release, and Twisted 2.5.0 had been out for a very long > time. > > Brian > +1 for requiring Twisted 8.0+ # Steve From laurent.dufrechou at gmail.com Fri Mar 28 20:52:30 2008 From: laurent.dufrechou at gmail.com (=?us-ascii?Q?Laurent_Dufrechou?=) Date: Sat, 29 Mar 2008 01:52:30 +0100 Subject: [IPython-dev] How to create a new ticket? Message-ID: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com> Hi guys, I've found a little feature request : 'a'.+TAB does not autocomplete. How can I put a ticket inside trac?? I've tried to register but I need a scipy user/password. Laurent -------------- next part -------------- An HTML attachment was scrubbed... URL: From vivainio at gmail.com Sat Mar 29 03:28:56 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sat, 29 Mar 2008 09:28:56 +0200 Subject: [IPython-dev] How to create a new ticket? In-Reply-To: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com> References: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com> Message-ID: <46cb515a0803290028r43c66c41xd9131a7d9d19aed1@mail.gmail.com> On Sat, Mar 29, 2008 at 2:52 AM, Laurent Dufrechou wrote: > How can I put a ticket inside trac?? I've tried to register but I need a > scipy user/password? Try putting it on launchpad - it's about time we get started using the "bugs" section :-) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From laurent.dufrechou at gmail.com Sat Mar 29 05:26:45 2008 From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=) Date: Sat, 29 Mar 2008 10:26:45 +0100 Subject: [IPython-dev] How to create a new ticket? In-Reply-To: <46cb515a0803290028r43c66c41xd9131a7d9d19aed1@mail.gmail.com> References: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com> <46cb515a0803290028r43c66c41xd9131a7d9d19aed1@mail.gmail.com> Message-ID: <47ee0b73.0b38560a.66cc.0e88@mx.google.com> Hum, i've gone to launchad, ipython project then bugs. And no bug tracker is defined. I think it's not defined for the moment :) Do I miss something? -----Message d'origine----- De?: Ville M. Vainio [mailto:vivainio at gmail.com] Envoy??: samedi 29 mars 2008 08:29 ??: Laurent Dufrechou Cc?: ipython-dev Mailing list Objet?: Re: [IPython-dev] How to create a new ticket? On Sat, Mar 29, 2008 at 2:52 AM, Laurent Dufrechou wrote: > How can I put a ticket inside trac?? I've tried to register but I need a > scipy user/password Try putting it on launchpad - it's about time we get started using the "bugs" section :-) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From laurent.dufrechou at gmail.com Sat Mar 29 07:02:52 2008 From: laurent.dufrechou at gmail.com (=?us-ascii?Q?Laurent_Dufrechou?=) Date: Sat, 29 Mar 2008 12:02:52 +0100 Subject: [IPython-dev] Help! TR: TR: Ipython plugin Message-ID: <47ee21fb.05a0660a.4a91.3ff1@mx.google.com> Hi guys, I've submitted a pre-plugin for editra as an alternative to pyshell ;) All is working except some quirk with ctrl+c (editra interact with it) and enter under mac osx.. (I had never tried under mac osx.) So here are my two help request: -Can somebody tell if it exist a macosx emulator??? Or is there someone willing to help me debug this remotely? (else trying to correct the code by itself with my help or perhaps via a vnc connection?) Here is what cody saw: 2) The plugin loads and shows on Mac OSX but there must be some problem with the key handling because hitting 'Enter' only inserts a new line in the control and doesn't execute the command. This problem doesn't exist when running under wxGTK. -Cody is asking if it is possible to embed Ipython in the plugin: It might also be interesting to try and embed IPython itself into the plugin so nobody has to worry about dependencies. If IPython is pure python (not a C extension) this should be as easy as copying the IPython directory into your plugins __init__.py path and then including it the setup.py's data files. Does Ipython as C dependency I'm not aware of??? Cheers, Laurent -------------- next part -------------- An HTML attachment was scrubbed... URL: From vivainio at gmail.com Sat Mar 29 07:05:45 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sat, 29 Mar 2008 13:05:45 +0200 Subject: [IPython-dev] How to create a new ticket? In-Reply-To: <47ee0b73.0b38560a.66cc.0e88@mx.google.com> References: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com> <46cb515a0803290028r43c66c41xd9131a7d9d19aed1@mail.gmail.com> <47ee0b73.0b38560a.66cc.0e88@mx.google.com> Message-ID: <46cb515a0803290405x6ab3b518g22e8107514f19cd9@mail.gmail.com> On Sat, Mar 29, 2008 at 11:26 AM, Laurent Dufr?chou wrote: > Hum, i've gone to launchad, ipython project then bugs. > And no bug tracker is defined. > I think it's not defined for the moment :) > Do I miss something? No, you didn't, I had to enable the bug tracking. Try again :-) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From laurent.dufrechou at gmail.com Sat Mar 29 07:29:04 2008 From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=) Date: Sat, 29 Mar 2008 12:29:04 +0100 Subject: [IPython-dev] How to create a new ticket? In-Reply-To: <46cb515a0803290405x6ab3b518g22e8107514f19cd9@mail.gmail.com> References: <47ed92ec.1f205e0a.1536.ffffb1d7@mx.google.com> <46cb515a0803290028r43c66c41xd9131a7d9d19aed1@mail.gmail.com> <47ee0b73.0b38560a.66cc.0e88@mx.google.com> <46cb515a0803290405x6ab3b518g22e8107514f19cd9@mail.gmail.com> Message-ID: <47ee281f.0ab6660a.7304.61f3@mx.google.com> Ok now we can report bug throught launchpad! You can now find a fix to my bug report :p By the way, how about old trac bug list? Will it be lost? -----Message d'origine----- De?: Ville M. Vainio [mailto:vivainio at gmail.com] Envoy??: samedi 29 mars 2008 12:06 ??: Laurent Dufr?chou Cc?: ipython-dev Mailing list Objet?: Re: [IPython-dev] How to create a new ticket? On Sat, Mar 29, 2008 at 11:26 AM, Laurent Dufr?chou wrote: > Hum, i've gone to launchad, ipython project then bugs. > And no bug tracker is defined. > I think it's not defined for the moment :) > Do I miss something? No, you didn't, I had to enable the bug tracking. Try again :-) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From barrywark at gmail.com Sat Mar 29 13:51:19 2008 From: barrywark at gmail.com (Barry Wark) Date: Sat, 29 Mar 2008 10:51:19 -0700 Subject: [IPython-dev] Twisted 8... In-Reply-To: <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org> <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> Message-ID: +1 On Fri, Mar 28, 2008 at 1:00 PM, Brian Granger wrote: > Glenn, > > I am plannig to get IPython1 working with Twisted 8.0 soon. As you > mention, there should only be a few things we have to do. How do > people feel about requiring the new version of Twisted? It is a > significant release, and Twisted 2.5.0 had been out for a very long > time. > > Brian > > > > On Fri, Mar 28, 2008 at 1:29 PM, Glenn H Tarbox, PhD wrote: > > Ipython1 no longer compiles with the latest Twisted trunk because of > > Twisted's new numbering scheme. Fails in the version check. > > > > I'm only aware of one significant IPython1 refactoring issue relating to > > the blockingThread capability now in the Twisted mainline. (probably not > > refactoring, just an adjustment to use Twisted's call rather than > > whatever version IPython1 was using in anticipation of this new twisted > > feature.) > > > > -glenn > > > > -- > > Glenn H. Tarbox, PhD | Don't worry about people stealing your ideas. If your ideas > > 206-494-0819 | are any good, you'll have to ram them down people's throats > > glenn at tarbox.org | ?Howard Aiken, IBM engineer > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ondrej at certik.cz Mon Mar 31 10:04:13 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Mon, 31 Mar 2008 16:04:13 +0200 Subject: [IPython-dev] GSoC project - a notebook like GUI Message-ID: <85b5c3130803310704g794d9085y84d037273bb42e09@mail.gmail.com> Hi, we got an interesting proposal for the Google Summer of Code (GSoC) in SymPy: http://groups.google.com/group/sympy/browse_thread/thread/f4a28675d34e0ee6 and an interesting discussion has taken place in the thread. With ipython1 and it's frontends in mind, I'd like to ask about your opinions of the GSoC proposal. Can maybe some work be done together with ipython1? Ondrej From bwaters at nrao.edu Mon Mar 31 13:48:27 2008 From: bwaters at nrao.edu (Boyd Waters) Date: Mon, 31 Mar 2008 11:48:27 -0600 Subject: [IPython-dev] Twisted 8... In-Reply-To: <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org> <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> Message-ID: On Mar 28, 2008, at 2:00 PM, Brian Granger wrote: > How do people feel about requiring the new version of Twisted? +1 no guts, no glory From fperez.net at gmail.com Mon Mar 31 20:15:58 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 31 Mar 2008 17:15:58 -0700 Subject: [IPython-dev] Twisted 8... In-Reply-To: <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org> <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> Message-ID: On Fri, Mar 28, 2008 at 1:00 PM, Brian Granger wrote: > I am plannig to get IPython1 working with Twisted 8.0 soon. As you > mention, there should only be a few things we have to do. How do > people feel about requiring the new version of Twisted? It is a > significant release, and Twisted 2.5.0 had been out for a very long > time. Just curious: what *benefits* do we obtain from using twisted 8.0? It's nice that today, all of ipython1's depenedencies are satisfied out of the box on current linux distros. But even the not-yet-released ubuntu has twisted 8, they are still on 2.5: http://packages.ubuntu.com/hardy/python-twisted-core I'm not fundamentally opposed to requiring 8.0 if we actually need it, but otherwise I'd prefer to find a way (if possible, I haven't looked at the code) to work with both 2.5 and 8.0. I hope the twisted devs didn't make so many backwards incompatible changes as to make that impossible... Cheers, f From gael.varoquaux at normalesup.org Mon Mar 31 20:18:18 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Tue, 1 Apr 2008 02:18:18 +0200 Subject: [IPython-dev] Twisted 8... In-Reply-To: References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org> <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> Message-ID: <20080401001817.GM26101@phare.normalesup.org> On Mon, Mar 31, 2008 at 05:15:58PM -0700, Fernando Perez wrote: > I'm not fundamentally opposed to requiring 8.0 if we actually need it, > but otherwise I'd prefer to find a way (if possible, I haven't looked > at the code) to work with both 2.5 and 8.0. +1. I have enough work like this without dealing with installing software myself. Cheers, Ga?l From barrywark at gmail.com Mon Mar 31 21:18:35 2008 From: barrywark at gmail.com (Barry Wark) Date: Mon, 31 Mar 2008 18:18:35 -0700 Subject: [IPython-dev] Twisted 8... In-Reply-To: References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org> <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> Message-ID: Being installable via easy_install is a huge win (really a requirement) on OS X 10.5. Leopard comes with a system Twisted 2.4 and installing a newer version (such as the 2.5 required by ipython1) requires mucking with the PYTHONPATH (not a mac user-friendly task). Even worse, PYTHONPATH changes are not propagated via sudo (for obvious reasons), so running code that requires twisted 2.5 as root is impossible, even with the PYTHONPATH modification. Installation via easy_install fixes these issues (the easy_install'd Twisted is placed before the system Twisted on the sys.path by setuptools). With setuptools/easy_install support, it's possible to automate deployment of pre-compiled eggs. There's currently a branch of Twisted 2.5 that is installable via setuptools/easy_install, but the official release isn't. So, making Twisted and ipython1 a reality of OS X will require setuptools/easy_install support. So, +1 from OS X land. Barry On Mon, Mar 31, 2008 at 5:15 PM, Fernando Perez wrote: > On Fri, Mar 28, 2008 at 1:00 PM, Brian Granger wrote: > > > > I am plannig to get IPython1 working with Twisted 8.0 soon. As you > > mention, there should only be a few things we have to do. How do > > people feel about requiring the new version of Twisted? It is a > > significant release, and Twisted 2.5.0 had been out for a very long > > time. > > Just curious: what *benefits* do we obtain from using twisted 8.0? > It's nice that today, all of ipython1's depenedencies are satisfied > out of the box on current linux distros. But even the > not-yet-released ubuntu has twisted 8, they are still on 2.5: > > http://packages.ubuntu.com/hardy/python-twisted-core > > I'm not fundamentally opposed to requiring 8.0 if we actually need it, > but otherwise I'd prefer to find a way (if possible, I haven't looked > at the code) to work with both 2.5 and 8.0. I hope the twisted devs > didn't make so many backwards incompatible changes as to make that > impossible... > > Cheers, > > > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From fperez.net at gmail.com Mon Mar 31 21:48:25 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 31 Mar 2008 18:48:25 -0700 Subject: [IPython-dev] Twisted 8... In-Reply-To: References: <1206732570.6425.9.camel@ghtmyth.hq.tarbox.org> <6ce0ac130803281300r65c305e8h40f70a95a3173539@mail.gmail.com> Message-ID: On Mon, Mar 31, 2008 at 6:18 PM, Barry Wark wrote: > Being installable via easy_install is a huge win (really a > requirement) on OS X 10.5. Thanks for the info, easy_install support is certainly a benefit. That doesn't outweigh the cost of losing out-of-the-box installability on all linuxes, I'm afraid (it's not our fault that OSX is such a mess to develop for when outside of the Apple garden), so we should find out what's needed to make the code run with both 2.5 and 8.0. I'm now praying that they didn't make it impossible... Brian, you know that code far better than I do. Any ideas on how hard that would be? Cheers, f