From vishal.vatsa at gmail.com Wed Nov 5 06:18:13 2008 From: vishal.vatsa at gmail.com (Vishal Vatsa) Date: Wed, 05 Nov 2008 11:18:13 +0000 Subject: [IPython-dev] ipcluster with clusterfile References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> Message-ID: Brian Granger wrote: >> We're at the RC1 level now, so I'd like to hold off on anything but >> really focused changes. If you think this requires major work, I'd >> say let's disable it altogether and document it, so we can cut the >> release quickly. > > This was my leaning as well. For this release, I am going to disable > the clusterfile and simply print a message to the user that it has > been disabled. > >> Once the release is out, we can open again a 'merge window' for more >> invasive changes. >> Guys, Is any one working on ipcluster? If not, could you explain the issues to me and I would love to have a go at the implementation. -vishal From ellisonbg.net at gmail.com Wed Nov 5 11:51:30 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 5 Nov 2008 08:51:30 -0800 Subject: [IPython-dev] ipcluster with clusterfile In-Reply-To: References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> Message-ID: <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> Vishal, Actually we have been working on ipcluster some. What aspect of ipcluster are you most interested in? (localhost on Windows, ssh cluster, PBS, etc) While our work is not done, I am more than willing to share the new code with you (I just need to push it up to our bazaar repos on launchpad) so you can help test it/work on it for your usage case. Cheers, Brian On Wed, Nov 5, 2008 at 3:18 AM, Vishal Vatsa wrote: > Brian Granger wrote: > >>> We're at the RC1 level now, so I'd like to hold off on anything but >>> really focused changes. If you think this requires major work, I'd >>> say let's disable it altogether and document it, so we can cut the >>> release quickly. >> >> This was my leaning as well. For this release, I am going to disable >> the clusterfile and simply print a message to the user that it has >> been disabled. >> >>> Once the release is out, we can open again a 'merge window' for more >>> invasive changes. >>> > Guys, > > Is any one working on ipcluster? > > If not, could you explain the issues to me and I would > love to have a go at the implementation. > > > -vishal > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From vishal.vatsa at gmail.com Wed Nov 5 12:13:21 2008 From: vishal.vatsa at gmail.com (Vishal Vatsa) Date: Wed, 05 Nov 2008 17:13:21 +0000 Subject: [IPython-dev] ipcluster with clusterfile References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> Message-ID: Brian, Thanks for the reply. I am interested in ssh cluster. Mainly I would like to use the cluster file functionality to describe my cluster and startup the controller/engines. I am more than happy to be a tester/helper, let me know what I can do to help. -vishal Brian Granger wrote: > Vishal, > > Actually we have been working on ipcluster some. What aspect of > ipcluster are you most interested in? (localhost on Windows, ssh > cluster, PBS, etc) > > While our work is not done, I am more than willing to share the new > code with you (I just need to push it up to our bazaar repos on > launchpad) so you can help test it/work on it for your usage case. > > Cheers, > > Brian > > On Wed, Nov 5, 2008 at 3:18 AM, Vishal Vatsa > wrote: >> Brian Granger wrote: >> >>>> We're at the RC1 level now, so I'd like to hold off on anything but >>>> really focused changes. If you think this requires major work, I'd >>>> say let's disable it altogether and document it, so we can cut the >>>> release quickly. >>> >>> This was my leaning as well. For this release, I am going to disable >>> the clusterfile and simply print a message to the user that it has >>> been disabled. >>> >>>> Once the release is out, we can open again a 'merge window' for more >>>> invasive changes. >>>> >> Guys, >> >> Is any one working on ipcluster? >> >> If not, could you explain the issues to me and I would >> love to have a go at the implementation. >> >> >> -vishal >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >> From vivainio at gmail.com Wed Nov 5 12:22:30 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 5 Nov 2008 19:22:30 +0200 Subject: [IPython-dev] Rethinking the gui (and qt ui) Message-ID: <46cb515a0811050922j18027169ve40f75e00dfeee45@mail.gmail.com> Instead of doing the qt gui the way wx guis are done (i.e. mirror the console ipython way, with autoindent), I thought of adding an interesting twist to multiline statements: - You can press ctrl+b to launch "block editing" mode - This is a normal qscintilla text control, with scintilla python lexer doing the coloring and ipython doing the tab completion - ctrl + enter sends the mulitine block to ipython - ctrl + pgup / pgdn brings entries from history to the editor This will allow us to get rid of all the complexity related to indentation, and editing of previous lines. We can push everything directly to ipython, since everyhing is a "complete statement". At some point, we may even play with the idea of "ipython lite" branch, where terminal/readline/autoindent/partial statement complexity is factored away (but which requires a gui). -- Ville M. Vainio http://tinyurl.com/vainio From ellisonbg.net at gmail.com Wed Nov 5 12:19:45 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 5 Nov 2008 09:19:45 -0800 Subject: [IPython-dev] ipcluster with clusterfile In-Reply-To: References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> Message-ID: <6ce0ac130811050919s1eae7fc8l9e8020ff4420c8db@mail.gmail.com> OK, I will keep you posted and get you code ASAP. It may be few days though as I am really busy on some other things through tomorrow. Brian On Wed, Nov 5, 2008 at 9:13 AM, Vishal Vatsa wrote: > Brian, > > Thanks for the reply. > > I am interested in ssh cluster. Mainly I would like to use > the cluster file functionality to describe my cluster and > startup the controller/engines. > > I am more than happy to be a tester/helper, let me know > what I can do to help. > > -vishal > > Brian Granger wrote: > >> Vishal, >> >> Actually we have been working on ipcluster some. What aspect of >> ipcluster are you most interested in? (localhost on Windows, ssh >> cluster, PBS, etc) >> >> While our work is not done, I am more than willing to share the new >> code with you (I just need to push it up to our bazaar repos on >> launchpad) so you can help test it/work on it for your usage case. >> >> Cheers, >> >> Brian >> >> On Wed, Nov 5, 2008 at 3:18 AM, Vishal Vatsa >> wrote: >>> Brian Granger wrote: >>> >>>>> We're at the RC1 level now, so I'd like to hold off on anything but >>>>> really focused changes. If you think this requires major work, I'd >>>>> say let's disable it altogether and document it, so we can cut the >>>>> release quickly. >>>> >>>> This was my leaning as well. For this release, I am going to disable >>>> the clusterfile and simply print a message to the user that it has >>>> been disabled. >>>> >>>>> Once the release is out, we can open again a 'merge window' for more >>>>> invasive changes. >>>>> >>> Guys, >>> >>> Is any one working on ipcluster? >>> >>> If not, could you explain the issues to me and I would >>> love to have a go at the implementation. >>> >>> >>> -vishal >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >>> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From vivainio at gmail.com Wed Nov 5 12:47:01 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 5 Nov 2008 19:47:01 +0200 Subject: [IPython-dev] Rethinking the gui (and qt ui) In-Reply-To: <46cb515a0811050922j18027169ve40f75e00dfeee45@mail.gmail.com> References: <46cb515a0811050922j18027169ve40f75e00dfeee45@mail.gmail.com> Message-ID: <46cb515a0811050947j1ee6fb15w32f0e0a89d350e83@mail.gmail.com> On Wed, Nov 5, 2008 at 7:22 PM, Ville M. Vainio wrote: > At some point, we may even play with the idea of "ipython lite" > branch, where terminal/readline/autoindent/partial statement > complexity is factored away (but which requires a gui). Interestingly, this allows trivial frontend / backend separtion: command blocks go to core (some of which are "invisible", e.g. completion requests), stream of text (that is converted to html and shown in frontend output area) comes back. It may even be possible to connect to a server running ipython via plain ssh - that is, we connect to server via ssh (initiated by frontend, using an ssh lib), launch ipython in "backend" mode (I'm not yet sure whether it needs to be a separate mode) on the server, and interact away. -- Ville M. Vainio http://tinyurl.com/vainio From barrywark at gmail.com Wed Nov 5 13:08:56 2008 From: barrywark at gmail.com (Barry Wark) Date: Wed, 5 Nov 2008 10:08:56 -0800 Subject: [IPython-dev] Rethinking the gui (and qt ui) In-Reply-To: <46cb515a0811050947j1ee6fb15w32f0e0a89d350e83@mail.gmail.com> References: <46cb515a0811050922j18027169ve40f75e00dfeee45@mail.gmail.com> <46cb515a0811050947j1ee6fb15w32f0e0a89d350e83@mail.gmail.com> Message-ID: Ville, This is the exact approach that I've taken with the frontend.asyncfrontendbase and the Cocoa frontend I'm developing. asyncfrontendbase takes entire blocks and callsback into the frontend when results/failures are available. My vision for this is that the UI will be more similar to Mathematica's notebook UI. Right now, I'm writing a Cocoa frontend using native widgets to display the blocks/results -- much as you seem to be suggesting. The other possibility is to use HTML and DOM manipulation to achieve the same UI. The advantage of this approach is that any UI kit with a browser widget (both Qt and Cocoa now support WebKit, for example) would have a "native" frontend. The downside is that the text editing "widgets" within a WebKit view may be more limited than those available natively from scintilla or Cocoa's NSTextView. Ultimately, I think the HTML/DOM approach will provide the maximum benefit for the maximal number of users but I have very little experience with HTML & DOM programming, so I've been wary of diving in. We should discuss this together since it sounds like we can benefit from each other's work greatly. I'm happy to walk through the frontend module with you, at your convenience. barry On Wed, Nov 5, 2008 at 9:47 AM, Ville M. Vainio wrote: > On Wed, Nov 5, 2008 at 7:22 PM, Ville M. Vainio wrote: > >> At some point, we may even play with the idea of "ipython lite" >> branch, where terminal/readline/autoindent/partial statement >> complexity is factored away (but which requires a gui). > > Interestingly, this allows trivial frontend / backend separtion: > command blocks go to core (some of which are "invisible", e.g. > completion requests), stream of text (that is converted to html and > shown in frontend output area) comes back. > > It may even be possible to connect to a server running ipython via > plain ssh - that is, we connect to server via ssh (initiated by > frontend, using an ssh lib), launch ipython in "backend" mode (I'm not > yet sure whether it needs to be a separate mode) on the server, and > interact away. > > -- > Ville M. Vainio > http://tinyurl.com/vainio > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From barrywark at gmail.com Thu Nov 13 00:28:26 2008 From: barrywark at gmail.com (Barry Wark) Date: Wed, 12 Nov 2008 21:28:26 -0800 Subject: [IPython-dev] [Bug 215994] Re: We need to add the "complete" method to the Interpreter/Engine/Controller in IPython1 In-Reply-To: <20081111194218.25167.96550.launchpad@gandwana.canonical.com> References: <20080411223442.10860.39471.malonedeb@gandwana.canonical.com> <20081111194218.25167.96550.launchpad@gandwana.canonical.com> Message-ID: I agree with the decision to push completion into the frontends. However, now we need to think about how the frontend is to gain knowlege of the relevant namespace (presumably the user wants the completion to be relevant for the namespace where the command will eventually be executed, not for the frontend). Observing namespace changes in the engine (via the as-yet-to-be-written distributed notification system; I think this will be quite easy building on the synchronous notification system already written) is one possibility. Querying the entire engine namespace after each executed block is an other (less appealing) option. Thoughts? Barry On Tue, Nov 11, 2008 at 11:42 AM, Brian Granger wrote: > ** Changed in: ipython > Status: New => Won't Fix > > -- > We need to add the "complete" method to the Interpreter/Engine/Controller in IPython1 > https://bugs.launchpad.net/bugs/215994 > You received this bug notification because you are a member of IPython > Developers, which is subscribed to IPython. > > Status in IPython - Enhanced Interactive Python: Won't Fix > > Bug description: > The Interpreter class in ipython1.core.interpreter should have a complete method that take a string and returns a valid list of completions. For now, this can just raise NotImplementedError. This method will also need to be added to the various Engine/Controller interfaces and implementations. This is needed so frontend writers can begin to call the method. > From vishal.vatsa at gmail.com Thu Nov 13 06:02:04 2008 From: vishal.vatsa at gmail.com (Vishal Vatsa) Date: Thu, 13 Nov 2008 11:02:04 +0000 Subject: [IPython-dev] ipcluster with clusterfile References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> <6ce0ac130811050919s1eae7fc8l9e8020ff4420c8db@mail.gmail.com> Message-ID: Hi Brian, just wondering if you had a chance to push the ipcluster code yet. Thanks, -vishal Brian Granger wrote: > OK, I will keep you posted and get you code ASAP. It may be few days > though as I am really busy on some other things through tomorrow. > > Brian > > On Wed, Nov 5, 2008 at 9:13 AM, Vishal Vatsa > wrote: >> Brian, >> >> Thanks for the reply. >> >> I am interested in ssh cluster. Mainly I would like to use >> the cluster file functionality to describe my cluster and >> startup the controller/engines. >> >> I am more than happy to be a tester/helper, let me know >> what I can do to help. >> >> -vishal >> >> Brian Granger wrote: >> >>> Vishal, >>> >>> Actually we have been working on ipcluster some. What aspect of >>> ipcluster are you most interested in? (localhost on Windows, ssh >>> cluster, PBS, etc) >>> >>> While our work is not done, I am more than willing to share the new >>> code with you (I just need to push it up to our bazaar repos on >>> launchpad) so you can help test it/work on it for your usage case. >>> >>> Cheers, >>> >>> Brian >>> >>> On Wed, Nov 5, 2008 at 3:18 AM, Vishal Vatsa >>> wrote: >>>> Brian Granger wrote: >>>> >>>>>> We're at the RC1 level now, so I'd like to hold off on anything but >>>>>> really focused changes. If you think this requires major work, I'd >>>>>> say let's disable it altogether and document it, so we can cut the >>>>>> release quickly. >>>>> >>>>> This was my leaning as well. For this release, I am going to disable >>>>> the clusterfile and simply print a message to the user that it has >>>>> been disabled. >>>>> >>>>>> Once the release is out, we can open again a 'merge window' for more >>>>>> invasive changes. >>>>>> >>>> Guys, >>>> >>>> Is any one working on ipcluster? >>>> >>>> If not, could you explain the issues to me and I would >>>> love to have a go at the implementation. >>>> >>>> >>>> -vishal >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >>>> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >> From matt.p.foster at gmail.com Thu Nov 13 09:17:18 2008 From: matt.p.foster at gmail.com (Matt Foster) Date: Thu, 13 Nov 2008 14:17:18 +0000 Subject: [IPython-dev] [TextMate] IPython TextMate Bundle Message-ID: <1dcb3f200811130617p421b9898ma37d9de907d11e6a@mail.gmail.com> Hi All, A similar mail has already been on the users mailing list, so my apologies if you've seen most of this before. I've started working on a TextMate bundle for IPython, based on the info on the Wiki [1], the aim is to produce a BSD licensed bundle which helps to integrate TextMate with IPython. I have set up a project on Github [2] which currently contains: * Some help, which doubles as the README * commands for running the current file / line / section in IPython (via applescript, and Terminal.app) * a basic language grammar for ipythonrc config files. The GitHub page contains the README file which has instructions on how to get GetBundles, which will allow you to install the bundle (but not track changes). Alternatively, if you have git, you can get the bundle using the following commands: cd "~/Library/Application Support/TextMate/Bundles" git clone git://github.com/mattfoster/ipython-tmbundle.git IPython.tmbundle osascript -e 'tell app "TextMate" to reload bundles' GitHub users can fork the project and make their own changes. I'd really love to hear any ideas, suggestions or feature requests people have, and I've been told by Fernando that it's ok to use this list for discussions, provided we prefix mail subjects with [TextMate]. Thanks, Matt [1]: http://ipython.scipy.org/moin/Cookbook/UsingIPythonWithTextMate [2]: http://github.com/mattfoster/ipython-tmbundle/ -- Matt Foster | http://my-mili.eu/matt From ellisonbg.net at gmail.com Thu Nov 13 21:40:10 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 13 Nov 2008 18:40:10 -0800 Subject: [IPython-dev] ipcluster with clusterfile In-Reply-To: References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> <6ce0ac130811050919s1eae7fc8l9e8020ff4420c8db@mail.gmail.com> Message-ID: <6ce0ac130811131840l493ad4b4sb443db9b5cae0286@mail.gmail.com> Not yet, but I am working hard on it. I plan on pushing it by early next week. Brian On Thu, Nov 13, 2008 at 3:02 AM, Vishal Vatsa wrote: > Hi Brian, > > just wondering if you had a chance to push the ipcluster code yet. > > Thanks, > -vishal > > Brian Granger wrote: > >> OK, I will keep you posted and get you code ASAP. It may be few days >> though as I am really busy on some other things through tomorrow. >> >> Brian >> >> On Wed, Nov 5, 2008 at 9:13 AM, Vishal Vatsa >> wrote: >>> Brian, >>> >>> Thanks for the reply. >>> >>> I am interested in ssh cluster. Mainly I would like to use >>> the cluster file functionality to describe my cluster and >>> startup the controller/engines. >>> >>> I am more than happy to be a tester/helper, let me know >>> what I can do to help. >>> >>> -vishal >>> >>> Brian Granger wrote: >>> >>>> Vishal, >>>> >>>> Actually we have been working on ipcluster some. What aspect of >>>> ipcluster are you most interested in? (localhost on Windows, ssh >>>> cluster, PBS, etc) >>>> >>>> While our work is not done, I am more than willing to share the new >>>> code with you (I just need to push it up to our bazaar repos on >>>> launchpad) so you can help test it/work on it for your usage case. >>>> >>>> Cheers, >>>> >>>> Brian >>>> >>>> On Wed, Nov 5, 2008 at 3:18 AM, Vishal Vatsa >>>> wrote: >>>>> Brian Granger wrote: >>>>> >>>>>>> We're at the RC1 level now, so I'd like to hold off on anything but >>>>>>> really focused changes. If you think this requires major work, I'd >>>>>>> say let's disable it altogether and document it, so we can cut the >>>>>>> release quickly. >>>>>> >>>>>> This was my leaning as well. For this release, I am going to disable >>>>>> the clusterfile and simply print a message to the user that it has >>>>>> been disabled. >>>>>> >>>>>>> Once the release is out, we can open again a 'merge window' for more >>>>>>> invasive changes. >>>>>>> >>>>> Guys, >>>>> >>>>> Is any one working on ipcluster? >>>>> >>>>> If not, could you explain the issues to me and I would >>>>> love to have a go at the implementation. >>>>> >>>>> >>>>> -vishal >>>>> >>>>> _______________________________________________ >>>>> IPython-dev mailing list >>>>> IPython-dev at scipy.org >>>>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >>>>> >>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >>> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From vivainio at gmail.com Fri Nov 14 11:44:37 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Fri, 14 Nov 2008 18:44:37 +0200 Subject: [IPython-dev] [Bug 215994] Re: We need to add the "complete" method to the Interpreter/Engine/Controller in IPython1 In-Reply-To: References: <20080411223442.10860.39471.malonedeb@gandwana.canonical.com> <20081111194218.25167.96550.launchpad@gandwana.canonical.com> Message-ID: <46cb515a0811140844q3b799182gefb8b572b0bd08ba@mail.gmail.com> On Thu, Nov 13, 2008 at 7:28 AM, Barry Wark wrote: > I agree with the decision to push completion into the frontends. > However, now we need to think about how the frontend is to gain > knowlege of the relevant namespace (presumably the user wants the > completion to be relevant for the namespace where the command will > eventually be executed, not for the frontend). Observing namespace I don't really see the point of doing it in the frontend, since it's clearly a "model" level feature, and can't be reliably done in frontend. What about custom completers etc.? -- Ville M. Vainio http://tinyurl.com/vainio From barrywark at gmail.com Fri Nov 14 12:31:51 2008 From: barrywark at gmail.com (Barry Wark) Date: Fri, 14 Nov 2008 09:31:51 -0800 Subject: [IPython-dev] [Bug 215994] Re: We need to add the "complete" method to the Interpreter/Engine/Controller in IPython1 In-Reply-To: <46cb515a0811140844q3b799182gefb8b572b0bd08ba@mail.gmail.com> References: <20080411223442.10860.39471.malonedeb@gandwana.canonical.com> <20081111194218.25167.96550.launchpad@gandwana.canonical.com> <46cb515a0811140844q3b799182gefb8b572b0bd08ba@mail.gmail.com> Message-ID: On Fri, Nov 14, 2008 at 8:44 AM, Ville M. Vainio wrote: > On Thu, Nov 13, 2008 at 7:28 AM, Barry Wark wrote: > >> I agree with the decision to push completion into the frontends. >> However, now we need to think about how the frontend is to gain >> knowlege of the relevant namespace (presumably the user wants the >> completion to be relevant for the namespace where the command will >> eventually be executed, not for the frontend). Observing namespace > > I don't really see the point of doing it in the frontend, since it's > clearly a "model" level feature, and can't be reliably done in > frontend. What about custom completers etc.? Because the engine may be remote, I believe the consensus was that no one wanted to take the chance that there would be no completions available due to network lag or outage. > > -- > Ville M. Vainio > http://tinyurl.com/vainio > From ellisonbg.net at gmail.com Fri Nov 14 16:30:22 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 14 Nov 2008 13:30:22 -0800 Subject: [IPython-dev] [Bug 215994] Re: We need to add the "complete" method to the Interpreter/Engine/Controller in IPython1 In-Reply-To: References: <20080411223442.10860.39471.malonedeb@gandwana.canonical.com> <20081111194218.25167.96550.launchpad@gandwana.canonical.com> <46cb515a0811140844q3b799182gefb8b572b0bd08ba@mail.gmail.com> Message-ID: <6ce0ac130811141330r7761775n7424191f06254333@mail.gmail.com> Sorry about all of this. When I marked this ticket as "Won't Fix", I did so for a very different reason. The main reason is that our ticket system had lots of stale tickets and this was one of them. In some cases I got rid of tickets that were "future work that we all know needs to get done eventually." So here is my intention and thoughts on the issue: * The complete method *should* be in the backend. * That doesn't prevent a given frontend from trying to implement complete in a different, more efficint manner. * This ticket is misleading as it will take a huge amount of refactoring in the old ipython core to implement this properly. We have *much* more to do that simply add the method. * I want our tickets to be specific and detailed. * This particular issue probably has 10 other things that need to be done first. Things like this should probably go into a development blueprint in our sphinx docs with notes on everything that needs to happen. But until someone has lots of time to put into the refactoring, I guess I would rather have our ticket list be more focused. Does this make sense? Cheers, Brian On Fri, Nov 14, 2008 at 9:31 AM, Barry Wark wrote: > On Fri, Nov 14, 2008 at 8:44 AM, Ville M. Vainio wrote: >> On Thu, Nov 13, 2008 at 7:28 AM, Barry Wark wrote: >> >>> I agree with the decision to push completion into the frontends. >>> However, now we need to think about how the frontend is to gain >>> knowlege of the relevant namespace (presumably the user wants the >>> completion to be relevant for the namespace where the command will >>> eventually be executed, not for the frontend). Observing namespace >> >> I don't really see the point of doing it in the frontend, since it's >> clearly a "model" level feature, and can't be reliably done in >> frontend. What about custom completers etc.? > > Because the engine may be remote, I believe the consensus was that no > one wanted to take the chance that there would be no completions > available due to network lag or outage. > >> >> -- >> Ville M. Vainio >> http://tinyurl.com/vainio >> > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Fri Nov 14 16:51:16 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 14 Nov 2008 13:51:16 -0800 Subject: [IPython-dev] Tweaking the IPython development workflow Message-ID: <6ce0ac130811141351w561c0b06o8303030a5b1badf@mail.gmail.com> Hello all, So, we have been using bzr and launchpad for IPython development for a few months now, and overall, I think it has been a huge improvement over svn. I think it is encouraging more people to contribute to the project, which is great. At the same time, we have been trying to move to a system where all changes are reviewed before they are merged into the main branch (lp:ipython). After talking to Fernando the other day about how our current model has been working, I wanted to suggest that we make a few minor changes to our workflow as it relates to code review and branch management. Code review ========== >From launchpad's perspective, a branch can be in one of three states: 1. No current proposal to merge/code review exists for a branch 2. A proposal to merge/code review exists. 3. A proposal to merge exists, but it has been "disabled" by marking it as "Work in Progress." I only recently learning about this feature in launchpad. When you want to resubmit a branch for review, you simply do "resubmit proposal", which takes it back to state 2. One thing that is really nice is that this "work in progress/resubmit" feature keeps track of all past merge proposals and their comments. Currently, we have 8 branches with reviews requested: https://code.launchpad.net/ipython/+activereviews However, many of these branches are not actually ready for review. This makes it very difficult for us to quickly see what reviews need to be done. So, here is what I propose: * If your branch is truly no longer needing a review, please delete the proposal to merge. * If your branch is temporarily not needing review, please mark it as "Work in Progress." This will tell the rest of us that we don't have to review it right now. * When your branch is truly ready for review please a) create a new proposal to merge if none exist or b) resubmit it if it is marked "work in progress" * After a review is complete and the branch is merged either a) mark the branch as "work in progress" if the branch will eventually need another review or b) mark the branch as merged to delete the proposal. This should make it much easier for all of us to see what needs to be reviewed. Branch cruft ========= There are a number of branches that look inactive or dead. If you have a branch is truly dead could you remove the branch or at least mark it as abandoned. Thanks. I think these things will make it easier for all of us to get things done with launchpad. Any thoughts? Feedback? Cheers, Brian From barrywark at gmail.com Fri Nov 14 17:14:35 2008 From: barrywark at gmail.com (Barry Wark) Date: Fri, 14 Nov 2008 14:14:35 -0800 Subject: [IPython-dev] [Bug 215994] Re: We need to add the "complete" method to the Interpreter/Engine/Controller in IPython1 In-Reply-To: <6ce0ac130811141330r7761775n7424191f06254333@mail.gmail.com> References: <20080411223442.10860.39471.malonedeb@gandwana.canonical.com> <20081111194218.25167.96550.launchpad@gandwana.canonical.com> <46cb515a0811140844q3b799182gefb8b572b0bd08ba@mail.gmail.com> <6ce0ac130811141330r7761775n7424191f06254333@mail.gmail.com> Message-ID: On Fri, Nov 14, 2008 at 1:30 PM, Brian Granger wrote: > Sorry about all of this. When I marked this ticket as "Won't Fix", I > did so for a very different reason. The main reason is that our > ticket system had lots of stale tickets and this was one of them. In > some cases I got rid of tickets that were "future work that we all > know needs to get done eventually." > > So here is my intention and thoughts on the issue: > > * The complete method *should* be in the backend. Oh, good! I appologize for abdicating my responsibilities as an ipython-dev citizen and agreeing to putting things in the frontend. I really do hope we can get completion into the core/engine and should've stuck to my guns and pushed for it. Mea culpa. > * That doesn't prevent a given frontend from trying to implement > complete in a different, more efficint manner. > * This ticket is misleading as it will take a huge amount of > refactoring in the old ipython core to implement this properly. We > have *much* more to do that simply add the method. > * I want our tickets to be specific and detailed. > * This particular issue probably has 10 other things that need to be done first. > > Things like this should probably go into a development blueprint in > our sphinx docs with notes on everything that needs to happen. But > until someone has lots of time to put into the refactoring, I guess I > would rather have our ticket list be more focused. > > Does this make sense? Absolutely. In my miniscule time to devote to fun development (ipython is definitely in that category), my priorities are 1) cocoa frontend 2) distributed notification 3) completion. If no one has taken on the blueprinting task by the time I get to 3 (some time around 2024 at the current rate), I'll make a start. Just to continue the discussion, off the top of my head I see two high-level design issues facing completion triggered by a frontend: 1. GUI toolkits often expect completion options to be returned to the toolkit and/or displayed synchronously. Getting them via a deferred is going to be tricky unless we can block until that deferred returns (with a time out, obviously). I'm pretty sure this is a solved issue (blocking until a deferred returns), but I know it can get tricky to get just right. 2. A use case where the frontend is fronting a controller with multiple engines. The user triggers completion. What options are returned? Since the engines may have different namespaces, it's unclear which namespace to use for completion. Will the frontend have to know which engine will receive the resulting block (i.e. the one the user is editing)? Cheers, Barry > > Cheers, > > Brian > > > On Fri, Nov 14, 2008 at 9:31 AM, Barry Wark wrote: >> On Fri, Nov 14, 2008 at 8:44 AM, Ville M. Vainio wrote: >>> On Thu, Nov 13, 2008 at 7:28 AM, Barry Wark wrote: >>> >>>> I agree with the decision to push completion into the frontends. >>>> However, now we need to think about how the frontend is to gain >>>> knowlege of the relevant namespace (presumably the user wants the >>>> completion to be relevant for the namespace where the command will >>>> eventually be executed, not for the frontend). Observing namespace >>> >>> I don't really see the point of doing it in the frontend, since it's >>> clearly a "model" level feature, and can't be reliably done in >>> frontend. What about custom completers etc.? >> >> Because the engine may be remote, I believe the consensus was that no >> one wanted to take the chance that there would be no completions >> available due to network lag or outage. >> >>> >>> -- >>> Ville M. Vainio >>> http://tinyurl.com/vainio >>> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >> > From barrywark at gmail.com Fri Nov 14 17:26:58 2008 From: barrywark at gmail.com (Barry Wark) Date: Fri, 14 Nov 2008 14:26:58 -0800 Subject: [IPython-dev] Tweaking the IPython development workflow In-Reply-To: <6ce0ac130811141351w561c0b06o8303030a5b1badf@mail.gmail.com> References: <6ce0ac130811141351w561c0b06o8303030a5b1badf@mail.gmail.com> Message-ID: On Fri, Nov 14, 2008 at 1:51 PM, Brian Granger wrote: > Hello all, > > So, we have been using bzr and launchpad for IPython development for a > few months now, and overall, I think it has been a huge improvement > over svn. I think it is encouraging more people to contribute to the > project, which is great. At the same time, we have been trying to > move to a system where all changes are reviewed before they are merged > into the main branch (lp:ipython). > > After talking to Fernando the other day about how our current model > has been working, I wanted to suggest that we make a few minor changes > to our workflow as it relates to code review and branch management. > > Code review > ========== > > >From launchpad's perspective, a branch can be in one of three states: > > 1. No current proposal to merge/code review exists for a branch > 2. A proposal to merge/code review exists. > 3. A proposal to merge exists, but it has been "disabled" by marking > it as "Work in Progress." I only recently learning about this feature > in launchpad. When you want to resubmit a branch for review, you > simply do "resubmit proposal", which takes it back to state 2. > > One thing that is really nice is that this "work in progress/resubmit" > feature keeps track of all past merge proposals and their comments. > > Currently, we have 8 branches with reviews requested: > > https://code.launchpad.net/ipython/+activereviews > > However, many of these branches are not actually ready for review. > This makes it very difficult for us to quickly see what reviews need > to be done. So, here is what I propose: > > * If your branch is truly no longer needing a review, please delete > the proposal to merge. > * If your branch is temporarily not needing review, please mark it as > "Work in Progress." This will tell the rest of us that we don't have > to review it right now. > * When your branch is truly ready for review please a) create a new > proposal to merge if none exist or b) resubmit it if it is marked > "work in progress" > * After a review is complete and the branch is merged either a) mark > the branch as "work in progress" if the branch will eventually need > another review or b) mark the branch as merged to delete the proposal. > > This should make it much easier for all of us to see what needs to be reviewed. > > Branch cruft > ========= > > There are a number of branches that look inactive or dead. If you > have a branch is truly dead could you remove the branch or at least > mark it as abandoned. Thanks. > > I think these things will make it easier for all of us to get things > done with launchpad. Any thoughts? Feedback? All of this makes great sense. The proposal/work in progress iteration seems like a great feature. One comment: do we have any method for assigning code reviews? By default, I suspect that you and Fernando and Min will be listed for many of the ipython1-related stuff (I listed you for the frontend branch merge, for example), but I know you all are busy. How about merge proposals also get an email to the list, requesting volunteers to do the review? In that vein, my lp:~barrywark/ipython/frontend branch is in need of review. It fixes the test code for the frontend package so that it plays correctly with trial/nose for all tests using Deferreds. Since I'm just learning Twisted's ins and outs, I'd appreciate someone running some eyes over the testing code. Any volunteers? barry > > Cheers, > > Brian > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Fri Nov 14 17:36:06 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 14 Nov 2008 14:36:06 -0800 Subject: [IPython-dev] Tweaking the IPython development workflow In-Reply-To: References: <6ce0ac130811141351w561c0b06o8303030a5b1badf@mail.gmail.com> Message-ID: <6ce0ac130811141436x682a103s17a943bf727bbd07@mail.gmail.com> I will definitely review your branch. Hopefully min could have a look as well. Brian On Fri, Nov 14, 2008 at 2:26 PM, Barry Wark wrote: > On Fri, Nov 14, 2008 at 1:51 PM, Brian Granger wrote: >> Hello all, >> >> So, we have been using bzr and launchpad for IPython development for a >> few months now, and overall, I think it has been a huge improvement >> over svn. I think it is encouraging more people to contribute to the >> project, which is great. At the same time, we have been trying to >> move to a system where all changes are reviewed before they are merged >> into the main branch (lp:ipython). >> >> After talking to Fernando the other day about how our current model >> has been working, I wanted to suggest that we make a few minor changes >> to our workflow as it relates to code review and branch management. >> >> Code review >> ========== >> >> >From launchpad's perspective, a branch can be in one of three states: >> >> 1. No current proposal to merge/code review exists for a branch >> 2. A proposal to merge/code review exists. >> 3. A proposal to merge exists, but it has been "disabled" by marking >> it as "Work in Progress." I only recently learning about this feature >> in launchpad. When you want to resubmit a branch for review, you >> simply do "resubmit proposal", which takes it back to state 2. >> >> One thing that is really nice is that this "work in progress/resubmit" >> feature keeps track of all past merge proposals and their comments. >> >> Currently, we have 8 branches with reviews requested: >> >> https://code.launchpad.net/ipython/+activereviews >> >> However, many of these branches are not actually ready for review. >> This makes it very difficult for us to quickly see what reviews need >> to be done. So, here is what I propose: >> >> * If your branch is truly no longer needing a review, please delete >> the proposal to merge. >> * If your branch is temporarily not needing review, please mark it as >> "Work in Progress." This will tell the rest of us that we don't have >> to review it right now. >> * When your branch is truly ready for review please a) create a new >> proposal to merge if none exist or b) resubmit it if it is marked >> "work in progress" >> * After a review is complete and the branch is merged either a) mark >> the branch as "work in progress" if the branch will eventually need >> another review or b) mark the branch as merged to delete the proposal. >> >> This should make it much easier for all of us to see what needs to be reviewed. >> >> Branch cruft >> ========= >> >> There are a number of branches that look inactive or dead. If you >> have a branch is truly dead could you remove the branch or at least >> mark it as abandoned. Thanks. >> >> I think these things will make it easier for all of us to get things >> done with launchpad. Any thoughts? Feedback? > > All of this makes great sense. The proposal/work in progress iteration > seems like a great feature. One comment: do we have any method for > assigning code reviews? By default, I suspect that you and Fernando > and Min will be listed for many of the ipython1-related stuff (I > listed you for the frontend branch merge, for example), but I know you > all are busy. How about merge proposals also get an email to the list, > requesting volunteers to do the review? > > In that vein, my lp:~barrywark/ipython/frontend branch is in need of > review. It fixes the test code for the frontend package so that it > plays correctly with trial/nose for all tests using Deferreds. Since > I'm just learning Twisted's ins and outs, I'd appreciate someone > running some eyes over the testing code. Any volunteers? > > barry > >> >> Cheers, >> >> Brian >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >> > From vivainio at gmail.com Sat Nov 15 14:03:53 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sat, 15 Nov 2008 21:03:53 +0200 Subject: [IPython-dev] Tweaking the IPython development workflow In-Reply-To: <6ce0ac130811141351w561c0b06o8303030a5b1badf@mail.gmail.com> References: <6ce0ac130811141351w561c0b06o8303030a5b1badf@mail.gmail.com> Message-ID: <46cb515a0811151103k487cb5eat303d232fe04c7f29@mail.gmail.com> On Fri, Nov 14, 2008 at 11:51 PM, Brian Granger wrote: > After talking to Fernando the other day about how our current model > has been working, I wanted to suggest that we make a few minor changes One could say the model could work better - last checkin to trunk was done 2 months ago (because people are not reviewing). Perhaps a "retroactive review" could work better, where we would just let the code be put to trunk and do bigger code reviews every now and then, in order to avoid stagnation. -- Ville M. Vainio http://tinyurl.com/vainio From vivainio at gmail.com Sat Nov 15 14:44:34 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sat, 15 Nov 2008 21:44:34 +0200 Subject: [IPython-dev] Tweaking the IPython development workflow In-Reply-To: <24433c3c0811151112r4582be26h1d1cde47f2086fa9@mail.gmail.com> References: <6ce0ac130811141351w561c0b06o8303030a5b1badf@mail.gmail.com> <46cb515a0811151103k487cb5eat303d232fe04c7f29@mail.gmail.com> <24433c3c0811151112r4582be26h1d1cde47f2086fa9@mail.gmail.com> Message-ID: <46cb515a0811151144x65ea1775xd4ce9f14a4c67021@mail.gmail.com> On Sat, Nov 15, 2008 at 9:12 PM, Yarko Tymciurak wrote: > I would encourage reviewing before checking into trunk - check into branches > in prep for review. > Many people (I included) take the trunk as quasi-stable, ready for "general > public" testing / feedback. Yes, that's what we are doing at the moment. So far, though, it hasn't been working too well. More specifically, if you find something that is obviously wrong in the trunk code and you have a trivial fix for it, it should be possible to check it to trunk with a quick schedule (as opposed to having people run broken code from trunk for months). I'm not saying we should change the process, though - just saying what I think would work better ;-). BTW, I also always recommend people to use the trunk version instead of the released versions, since trunk is almost always less broken than the older releases. Keeping the trunk "alive" still makes sense to me though - if integrate the branches too late, the code will not get excercised enough before the release (to state the obvious). -- Ville M. Vainio http://tinyurl.com/vainio From ellisonbg.net at gmail.com Sat Nov 15 18:19:38 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sat, 15 Nov 2008 15:19:38 -0800 Subject: [IPython-dev] Tweaking the IPython development workflow In-Reply-To: <46cb515a0811151103k487cb5eat303d232fe04c7f29@mail.gmail.com> References: <6ce0ac130811141351w561c0b06o8303030a5b1badf@mail.gmail.com> <46cb515a0811151103k487cb5eat303d232fe04c7f29@mail.gmail.com> Message-ID: <6ce0ac130811151519l6646a201u4a27dc63c65984ab@mail.gmail.com> > One could say the model could work better - last checkin to trunk was > done 2 months ago (because people are not reviewing). Perhaps a > "retroactive review" could work better, where we would just let the > code be put to trunk and do bigger code reviews every now and then, in > order to avoid stagnation. I agree that something looks off when trunk has been so inactive. However, from my perspective the cause is not merely the fact that we are now doing code review. My interpretation is rather thus: * I know a number of us have been focusing on other things (i.e., the rest of life). For example, most of our branches have been pretty quite during this time as well. If all of us had been doing lots of development in branches and all of it was waiting for review, then I would possibly agree that code review was a bottleneck. Also, our relative inactivity is also seen in our mailing list activity. Simply relaxing our code review won't help any of these things. * We are all new at code review and still getting used to it. A huge thing for me is that it is really easy for me to see what needs to be reviewed an to do the reviews. I think a few minor changes to our workflow will help in this area. * We all tend to do work in big batches = big changes, big reviews, big merges. I think our workflow would improve greatly if we would force ourselves to make much smaller changes and merge more often. I have been working with the sympy devs recently and this is one thing I have learned from them = code, review, merge often. I am just as guilty as anyone else of this. So..., I am willing to spend some time doing some code review, but before I do, I would like people to make sure their branch is _really_ ready for review if it has a proposal to merge. Otherwise, delete the proposal or mark it as "work in progress." So... 1) It would be great if people (myself included) could start doing smaller, more often code/review/merge cycles. We don't want to overdo this (like 1 commit per merge), but only merging every two months is too long. 2) Let's clean up the proposals to merge and then start actually doing reviews + merging. Here are the current people with branches marked for review/merging: Barry Wark, 1 branch (I think this actually needs review) Ville, 3 branches, 1 of which I think needs review Gael, 1 branch (does it need it?) Fernando, 2 branches (do they need it?) Cheers, Brian > -- > Ville M. Vainio > http://tinyurl.com/vainio > From vivainio at gmail.com Sun Nov 16 03:41:38 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sun, 16 Nov 2008 10:41:38 +0200 Subject: [IPython-dev] Tweaking the IPython development workflow In-Reply-To: <6ce0ac130811151519l6646a201u4a27dc63c65984ab@mail.gmail.com> References: <6ce0ac130811141351w561c0b06o8303030a5b1badf@mail.gmail.com> <46cb515a0811151103k487cb5eat303d232fe04c7f29@mail.gmail.com> <6ce0ac130811151519l6646a201u4a27dc63c65984ab@mail.gmail.com> Message-ID: <46cb515a0811160041v48a58eabh421fdc0691d41251@mail.gmail.com> On Sun, Nov 16, 2008 at 1:19 AM, Brian Granger wrote: > Here are the current people with branches marked for review/merging: > > Barry Wark, 1 branch (I think this actually needs review) > Ville, 3 branches, 1 of which I think needs review These need review for me: lp:~villemvainio/ipython/trunk-dev lp:~villemvainio/ipython/ip0-unittests-1 -- Ville M. Vainio http://tinyurl.com/vainio From ellisonbg.net at gmail.com Sun Nov 16 17:40:35 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sun, 16 Nov 2008 14:40:35 -0800 Subject: [IPython-dev] Tweaking the IPython development workflow In-Reply-To: <46cb515a0811160041v48a58eabh421fdc0691d41251@mail.gmail.com> References: <6ce0ac130811141351w561c0b06o8303030a5b1badf@mail.gmail.com> <46cb515a0811151103k487cb5eat303d232fe04c7f29@mail.gmail.com> <6ce0ac130811151519l6646a201u4a27dc63c65984ab@mail.gmail.com> <46cb515a0811160041v48a58eabh421fdc0691d41251@mail.gmail.com> Message-ID: <6ce0ac130811161440r1829737bhb15cd9d75bbf854e@mail.gmail.com> OK great, can you remove the review of the 3rd branch? I am going to try to get some review in this week. Brian On Sun, Nov 16, 2008 at 12:41 AM, Ville M. Vainio wrote: > On Sun, Nov 16, 2008 at 1:19 AM, Brian Granger wrote: > >> Here are the current people with branches marked for review/merging: >> >> Barry Wark, 1 branch (I think this actually needs review) >> Ville, 3 branches, 1 of which I think needs review > > These need review for me: > > lp:~villemvainio/ipython/trunk-dev > lp:~villemvainio/ipython/ip0-unittests-1 > > -- > Ville M. Vainio > http://tinyurl.com/vainio > From ellisonbg.net at gmail.com Sun Nov 16 17:46:30 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sun, 16 Nov 2008 14:46:30 -0800 Subject: [IPython-dev] ipcluster with clusterfile In-Reply-To: References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> <6ce0ac130811050919s1eae7fc8l9e8020ff4420c8db@mail.gmail.com> Message-ID: <6ce0ac130811161446y52aa965fx3394b0a40677a67c@mail.gmail.com> Vishal, OK, I just commited the new ipcluster to my launcpad branch here: https://code.launchpad.net/~ellisonbg/ipython/trunk-dev Please have a look and hack away. A few notes though: * For now, we are assuming in ipcluster that all the nodes have a shared filesystem. * We are using Twisted for all the process management things. Here are the docs on that part of Twisted. http://twistedmatrix.com/projects/core/documentation/howto/process.html Let me know if you have questions about twisted. * Currently the new version supports 3 cluster types (localhost, mpirun and pbs) * We would love to have an ssh cluster as well and any help you want to give in this area would be welcomed. * We are planning on refactoring our config system. This will change many things in ipcluster, but for the better. But, this won't happen overnight. I just wanted to give you the heads up that thingsa are still changing a low. * Some things in ipclsuter need to be refactored as well and tests need to be added. * I am not sure how soon these changes will be merged into trunk. So, for now I would work off my branch. If you start hacking on the ssh cluster, keep in touch so we can coordinate. Cheers, Brian On Thu, Nov 13, 2008 at 3:02 AM, Vishal Vatsa wrote: > Hi Brian, > > just wondering if you had a chance to push the ipcluster code yet. > > Thanks, > -vishal > > Brian Granger wrote: > >> OK, I will keep you posted and get you code ASAP. It may be few days >> though as I am really busy on some other things through tomorrow. >> >> Brian >> >> On Wed, Nov 5, 2008 at 9:13 AM, Vishal Vatsa >> wrote: >>> Brian, >>> >>> Thanks for the reply. >>> >>> I am interested in ssh cluster. Mainly I would like to use >>> the cluster file functionality to describe my cluster and >>> startup the controller/engines. >>> >>> I am more than happy to be a tester/helper, let me know >>> what I can do to help. >>> >>> -vishal >>> >>> Brian Granger wrote: >>> >>>> Vishal, >>>> >>>> Actually we have been working on ipcluster some. What aspect of >>>> ipcluster are you most interested in? (localhost on Windows, ssh >>>> cluster, PBS, etc) >>>> >>>> While our work is not done, I am more than willing to share the new >>>> code with you (I just need to push it up to our bazaar repos on >>>> launchpad) so you can help test it/work on it for your usage case. >>>> >>>> Cheers, >>>> >>>> Brian >>>> >>>> On Wed, Nov 5, 2008 at 3:18 AM, Vishal Vatsa >>>> wrote: >>>>> Brian Granger wrote: >>>>> >>>>>>> We're at the RC1 level now, so I'd like to hold off on anything but >>>>>>> really focused changes. If you think this requires major work, I'd >>>>>>> say let's disable it altogether and document it, so we can cut the >>>>>>> release quickly. >>>>>> >>>>>> This was my leaning as well. For this release, I am going to disable >>>>>> the clusterfile and simply print a message to the user that it has >>>>>> been disabled. >>>>>> >>>>>>> Once the release is out, we can open again a 'merge window' for more >>>>>>> invasive changes. >>>>>>> >>>>> Guys, >>>>> >>>>> Is any one working on ipcluster? >>>>> >>>>> If not, could you explain the issues to me and I would >>>>> love to have a go at the implementation. >>>>> >>>>> >>>>> -vishal >>>>> >>>>> _______________________________________________ >>>>> IPython-dev mailing list >>>>> IPython-dev at scipy.org >>>>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >>>>> >>> >>> >>> _______________________________________________ >>> IPython-dev mailing list >>> IPython-dev at scipy.org >>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >>> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Sun Nov 16 17:54:26 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sun, 16 Nov 2008 14:54:26 -0800 Subject: [IPython-dev] New ipcluster in ellisonbg's trunk-dev Message-ID: <6ce0ac130811161454x251e2d7fhb3153b99cbd0f365@mail.gmail.com> Hi, After a month or so of hacking on and off, I have finished a new version of the ipcluster script. This version has lots of new stuff: * Cross platform (Win32!) * Much more robust design (using Twisted) * Uses subcommands (ipcluster local, ipcluster pbs, etc.) for different cluster types. * Supports 3 cluster types currently (localhost, pbs, mpirun). The following just work: ipcluster local -n 4 ipcluster mpirun -n 16 --mpi-mpi4py ipcluster pbs -n 128 --pbs-script=pbs.template But, I am not sure how soon we want to merge this into trunk: * There are _no_ tests - this type of thing is very difficult to test. * No docs on this yet ( that is easy to fix). * The design is gong to change in massive ways - we need to refactor the config system in significant ways and that will change the API of ipcluster. * ipcluster itself needs some refactoring. It turns out that I will need to begin using Barry's notification center to really get the design right. But.... The ipcluster that we shipped with 0.9.1 is very broken. So...how do people think we should handle the review and merging of this? For now though, you will have to grab my branch to get this. Cheers, Brian From vishal.vatsa at gmail.com Mon Nov 17 04:41:54 2008 From: vishal.vatsa at gmail.com (Vishal Vatsa) Date: Mon, 17 Nov 2008 09:41:54 +0000 Subject: [IPython-dev] ipcluster with clusterfile References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> <6ce0ac130811050919s1eae7fc8l9e8020ff4420c8db@mail.gmail.com> <6ce0ac130811161446y52aa965fx3394b0a40677a67c@mail.gmail.com> Message-ID: Brian, Cool, thanks for this. I will have a look now. -vishal Brian Granger wrote: > Vishal, > > OK, I just commited the new ipcluster to my launcpad branch here: > > https://code.launchpad.net/~ellisonbg/ipython/trunk-dev > > Please have a look and hack away. A few notes though: > > * For now, we are assuming in ipcluster that all the nodes have a > shared filesystem. > > * We are using Twisted for all the process management things. Here > are the docs on that part of Twisted. > > http://twistedmatrix.com/projects/core/documentation/howto/process.html > > Let me know if you have questions about twisted. > > * Currently the new version supports 3 cluster types (localhost, mpirun > and pbs) > > * We would love to have an ssh cluster as well and any help you want > to give in this area would be welcomed. > > * We are planning on refactoring our config system. This will change > many things in ipcluster, but for the better. But, this won't happen > overnight. I just wanted to give you the heads up that thingsa are > still changing a low. > > * Some things in ipclsuter need to be refactored as well and tests > need to be added. > > * I am not sure how soon these changes will be merged into trunk. So, > for now I would work off my branch. > > If you start hacking on the ssh cluster, keep in touch so we can > coordinate. > > Cheers, > > Brian > > On Thu, Nov 13, 2008 at 3:02 AM, Vishal Vatsa > wrote: >> Hi Brian, >> >> just wondering if you had a chance to push the ipcluster code yet. >> >> Thanks, >> -vishal >> >> Brian Granger wrote: >> >>> OK, I will keep you posted and get you code ASAP. It may be few days >>> though as I am really busy on some other things through tomorrow. >>> >>> Brian >>> >>> On Wed, Nov 5, 2008 at 9:13 AM, Vishal Vatsa >>> wrote: >>>> Brian, >>>> >>>> Thanks for the reply. >>>> >>>> I am interested in ssh cluster. Mainly I would like to use >>>> the cluster file functionality to describe my cluster and >>>> startup the controller/engines. >>>> >>>> I am more than happy to be a tester/helper, let me know >>>> what I can do to help. >>>> >>>> -vishal >>>> >>>> Brian Granger wrote: >>>> >>>>> Vishal, >>>>> >>>>> Actually we have been working on ipcluster some. What aspect of >>>>> ipcluster are you most interested in? (localhost on Windows, ssh >>>>> cluster, PBS, etc) >>>>> >>>>> While our work is not done, I am more than willing to share the new >>>>> code with you (I just need to push it up to our bazaar repos on >>>>> launchpad) so you can help test it/work on it for your usage case. >>>>> >>>>> Cheers, >>>>> >>>>> Brian >>>>> >>>>> On Wed, Nov 5, 2008 at 3:18 AM, Vishal Vatsa >>>>> wrote: >>>>>> Brian Granger wrote: >>>>>> >>>>>>>> We're at the RC1 level now, so I'd like to hold off on anything but >>>>>>>> really focused changes. If you think this requires major work, I'd >>>>>>>> say let's disable it altogether and document it, so we can cut the >>>>>>>> release quickly. >>>>>>> >>>>>>> This was my leaning as well. For this release, I am going to >>>>>>> disable the clusterfile and simply print a message to the user that >>>>>>> it has been disabled. >>>>>>> >>>>>>>> Once the release is out, we can open again a 'merge window' for >>>>>>>> more invasive changes. >>>>>>>> >>>>>> Guys, >>>>>> >>>>>> Is any one working on ipcluster? >>>>>> >>>>>> If not, could you explain the issues to me and I would >>>>>> love to have a go at the implementation. >>>>>> >>>>>> >>>>>> -vishal >>>>>> >>>>>> _______________________________________________ >>>>>> IPython-dev mailing list >>>>>> IPython-dev at scipy.org >>>>>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >>>>>> >>>> >>>> >>>> _______________________________________________ >>>> IPython-dev mailing list >>>> IPython-dev at scipy.org >>>> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >>>> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev >> From matt.p.foster at gmail.com Mon Nov 17 07:04:53 2008 From: matt.p.foster at gmail.com (Matt Foster) Date: Mon, 17 Nov 2008 12:04:53 +0000 Subject: [IPython-dev] ipy_render Message-ID: <1dcb3f200811170404v137ced58i247aba04949b5670@mail.gmail.com> I was looking at the extensions and found that ipy_render only works on windows. I added support for darwin and linux (the two platforms I have access to ATM), and it would be easy to add more. I've attached the new version in case anyone finds it useful. Cheers, Matt -- Matt Foster | http://hackerific.net -------------- next part -------------- A non-text attachment was scrubbed... Name: ipy_render.py Type: text/x-python-script Size: 2343 bytes Desc: not available URL: From vivainio at gmail.com Mon Nov 17 15:30:01 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 17 Nov 2008 22:30:01 +0200 Subject: [IPython-dev] ipy_render In-Reply-To: <1dcb3f200811170404v137ced58i247aba04949b5670@mail.gmail.com> References: <1dcb3f200811170404v137ced58i247aba04949b5670@mail.gmail.com> Message-ID: <46cb515a0811171230p47988a0w97e5527c1646dac2@mail.gmail.com> On Mon, Nov 17, 2008 at 2:04 PM, Matt Foster wrote: > I was looking at the extensions and found that ipy_render only works > on windows. I added support for darwin and linux (the two platforms I > have access to ATM), and it would be easy to add more. > > I've attached the new version in case anyone finds it useful. It is useful - however, as such I cannot commit it because: - the toclip function belongs to platutils_*.py (now that multiple platform implemetations exist, i.e it's not win32-only extension) - It seems fishy to echo the string and pipe it to xsel (does it really work with many lines?). Rather, it should do os.popen("xsel -i","w").write(string) or somesuch. So, if you could polish this along these lines, it would be great and I would commit it. Not being able to write unit tests is understandable. Thanks for the contribution in any case, and keep us posted whether or not you are able to improve the contribution! -- Ville M. Vainio http://tinyurl.com/vainio From matt.p.foster at gmail.com Tue Nov 18 05:25:42 2008 From: matt.p.foster at gmail.com (Matt Foster) Date: Tue, 18 Nov 2008 10:25:42 +0000 Subject: [IPython-dev] ipy_render In-Reply-To: <46cb515a0811171230p47988a0w97e5527c1646dac2@mail.gmail.com> References: <1dcb3f200811170404v137ced58i247aba04949b5670@mail.gmail.com> <46cb515a0811171230p47988a0w97e5527c1646dac2@mail.gmail.com> Message-ID: <1dcb3f200811180225u40a6dd6bjec785440462d4107@mail.gmail.com> On Mon, Nov 17, 2008 at 8:30 PM, Ville M. Vainio wrote: > On Mon, Nov 17, 2008 at 2:04 PM, Matt Foster wrote: > >> I was looking at the extensions and found that ipy_render only works >> on windows. I added support for darwin and linux (the two platforms I >> have access to ATM), and it would be easy to add more. >> >> I've attached the new version in case anyone finds it useful. > > It is useful - however, as such I cannot commit it because: > > - the toclip function belongs to platutils_*.py (now that multiple > platform implemetations exist, i.e it's not win32-only extension) > > - It seems fishy to echo the string and pipe it to xsel (does it > really work with many lines?). Rather, it should do os.popen("xsel > -i","w").write(string) or somesuch. > > So, if you could polish this along these lines, it would be great and > I would commit it. Not being able to write unit tests is > understandable. > > Thanks for the contribution in any case, and keep us posted whether or > not you are able to improve the contribution! Thanks for the reply. I'm happy to improve the submission and send it again. I'm still new enough to python that I'm not sure what the best way of doing some things is, but I'm happy to go back and fix things if needs be. I'm just grabbing bazaar, so I should be able to send you a patch next time, too. Cheers, Matt -- Matt Foster | http://hackerific.net From matt.p.foster at gmail.com Tue Nov 18 06:35:47 2008 From: matt.p.foster at gmail.com (Matt Foster) Date: Tue, 18 Nov 2008 11:35:47 +0000 Subject: [IPython-dev] ipy_render In-Reply-To: <1dcb3f200811180225u40a6dd6bjec785440462d4107@mail.gmail.com> References: <1dcb3f200811170404v137ced58i247aba04949b5670@mail.gmail.com> <46cb515a0811171230p47988a0w97e5527c1646dac2@mail.gmail.com> <1dcb3f200811180225u40a6dd6bjec785440462d4107@mail.gmail.com> Message-ID: <1dcb3f200811180335s69acbbdbt770cd3c47e6fef4@mail.gmail.com> On Tue, Nov 18, 2008 at 10:25 AM, Matt Foster wrote: > > I'm happy to improve the submission and send it again. > Hi Again, I know it hasn't been long since I sent the last email but I've been playing around, and made a patch against lp:ipython. The main changes are: * Added read_from_cmd and write_to_cmd to platutils_posix. * Added check_cmd which uses read_from_cmd to run which and check the output status. * Added topclip functions for systems with pbcopy (mac) and xsel. * Moved the win32 toclip stuff from ipy_render into platutils_win32. * Finally, I modified the ipy_render file to use the new functions. I've tested this on my intel mac and linux boxes, but don't have any windows machines handy, so I can't check that side of things. I'm happy to make any more changes you think might be needed, and would appreciate any more feedback. See attached. Cheers, Matt -- Matt Foster | http://hackerific.net -------------- next part -------------- A non-text attachment was scrubbed... Name: ipy_render_and_platutils.diff Type: application/octet-stream Size: 4647 bytes Desc: not available URL: From vivainio at gmail.com Tue Nov 18 11:46:43 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Tue, 18 Nov 2008 18:46:43 +0200 Subject: [IPython-dev] ipy_render In-Reply-To: <1dcb3f200811180335s69acbbdbt770cd3c47e6fef4@mail.gmail.com> References: <1dcb3f200811170404v137ced58i247aba04949b5670@mail.gmail.com> <46cb515a0811171230p47988a0w97e5527c1646dac2@mail.gmail.com> <1dcb3f200811180225u40a6dd6bjec785440462d4107@mail.gmail.com> <1dcb3f200811180335s69acbbdbt770cd3c47e6fef4@mail.gmail.com> Message-ID: <46cb515a0811180846s1122e895l2cd4b20788f5ee7c@mail.gmail.com> On Tue, Nov 18, 2008 at 1:35 PM, Matt Foster wrote: > I've tested this on my intel mac and linux boxes, but don't have any > windows machines handy, so I can't check that side of things. > > I'm happy to make any more changes you think might be needed, and > would appreciate any more feedback. Did you try with os.popen() already? It should eliminate the need for read_*_command. -- Ville M. Vainio http://tinyurl.com/vainio From matt.p.foster at gmail.com Tue Nov 18 17:59:08 2008 From: matt.p.foster at gmail.com (Matt Foster) Date: Tue, 18 Nov 2008 22:59:08 +0000 Subject: [IPython-dev] [TextMate] IPython Bundle Demo Screencast Message-ID: <1dcb3f200811181459m60d6df71k83f7edfbd98cf358@mail.gmail.com> Hello, I'm pleased to announce that the IPython TextMate bundle is in a state where it can actually be used. To celebrate this, I've made a simple screencast demoing some of the basic features. You can find it at [1] , it should be ready to view fairly soon. The main features of note are: * Communication with IPython via the extension ipy_vimserver. This means the bundle should work right out of the box: just run import ipy_vimserver; ipy_vimserver.setup('session'), and you can connect. * running the current file in IPython * running the current line / selection in IPython * running the current file / selection / line in IPython with the profiler enabled * growl notification of salient points * rudimentary syntax highlighting of ipythonrc files, including warnings about broken comments * commands for editing the ipythonrc file and .ipython directory * built in help * access from a single key binding. The bundle can be downloaded from [2] or installed via GetBundles. For installation help, see [2]. We'd love to hear your feedback, so please send comments, suggestions and feedback to either the ipython-dev list (please prefix the subject with [TextMate]), or the ipython-tmbundle google group (just reply to this email). Thanks! [1]: http://www.vimeo.com/2281439 [2]: http://github.com/mattfoster/ipython-tmbundle/ -- Matt Foster | http://hackerific.net From vishal.vatsa at gmail.com Wed Nov 19 11:10:22 2008 From: vishal.vatsa at gmail.com (Vishal Vatsa) Date: Wed, 19 Nov 2008 16:10:22 +0000 Subject: [IPython-dev] ipcluster with clusterfile References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> <6ce0ac130811050919s1eae7fc8l9e8020ff4420c8db@mail.gmail.com> <6ce0ac130811161446y52aa965fx3394b0a40677a67c@mail.gmail.com> Message-ID: Brian Granger wrote: > If you start hacking on the ssh cluster, keep in touch so we can > coordinate. I am starting to have a look at this now, if you have any suggestions/issues that I should be aware of, please let me know. Other than that, I will ping you as soon as I have some thing to show. Regards, -vishal From ellisonbg.net at gmail.com Wed Nov 19 14:33:21 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 19 Nov 2008 11:33:21 -0800 Subject: [IPython-dev] ipcluster with clusterfile In-Reply-To: References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> <6ce0ac130811050919s1eae7fc8l9e8020ff4420c8db@mail.gmail.com> <6ce0ac130811161446y52aa965fx3394b0a40677a67c@mail.gmail.com> Message-ID: <6ce0ac130811191133q72bb2dc2wf03367d8c63879b@mail.gmail.com> >> If you start hacking on the ssh cluster, keep in touch so we can >> coordinate. > I am starting to have a look at this now, if you have any suggestions/issues that I should be aware of, please let me know. A couple of things: * I think the biggest thing we keep running into is in managing the furl files that the controller creates. You might want to read up on this topic in the ipython sphinx docs. Currently we are handling this by assuming that the hosts have a shared file system and that the ipython directory (to which the furl files are written is on that shared file system). But, we could instead manually move the furl files around using scp and this might make sense for the ssh cluster. * I would have a look at how the old ipcluster worked with ssh. There is a trick that is needed to handle environment variables. This can be found in the ipython trunk on launchpad. > Other than that, I will ping you as soon as I have some thing to show. Great! Don't hesitate to ask questions along the way. We really appreciate your willingness to work on this. Brian > > Regards, > -vishal > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Wed Nov 19 23:45:40 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 19 Nov 2008 20:45:40 -0800 Subject: [IPython-dev] IPython proposals to merge are cleaned up Message-ID: <6ce0ac130811192045re5723b8q6648c52252a8f078@mail.gmail.com> Hello all, I just went through the IPython proposals to merge on launchpad. There were a number of branches that I was unsure if they were ready for review. For those branches I changed their status to "work in progress." If you have one of these branches and it really is ready to be reviewed and merged, please change its status back to "ready for review." There are currently 4 branches that are indeed ready for review. Let's jump on these and get them reviewed so we can start merging again. Here they are: https://code.launchpad.net/ipython/+activereviews Cheers, Brian From bjracine at glosten.com Thu Nov 20 17:51:09 2008 From: bjracine at glosten.com (Benjamin J. Racine) Date: Thu, 20 Nov 2008 14:51:09 -0800 Subject: [IPython-dev] [TextMate] IPython Bundle to E-texteditor github port In-Reply-To: <1dcb3f200811181459m60d6df71k83f7edfbd98cf358@mail.gmail.com> Message-ID: <8C2B20C4348091499673D86BF10AB6762407BB0D@clipper.glosten.local> Matt and all, http://github.com/gtcaz/ebundles/tree/master/Bundles Have you noticed that this is where things seem to get "ported" over to e-texteditor? I could be a tester from the e-texteditor side. Further, I've added some snippets to my most recent download of the bundle. I just need to figure out how to get them up this evening onto a new branch on github. I'm sure it's easy, I just haven't taken the time yet to figure it out yet. Ben -----Original Message----- From: ipython-tmbundle at googlegroups.com [mailto:ipython-tmbundle at googlegroups.com] On Behalf Of Matt Foster Sent: Tuesday, November 18, 2008 2:59 PM To: ipython-tmbundle at googlegroups.com; ipython-dev at scipy.net Subject: [TextMate] IPython Bundle Demo Screencast Hello, I'm pleased to announce that the IPython TextMate bundle is in a state where it can actually be used. To celebrate this, I've made a simple screencast demoing some of the basic features. You can find it at [1] , it should be ready to view fairly soon. The main features of note are: * Communication with IPython via the extension ipy_vimserver. This means the bundle should work right out of the box: just run import ipy_vimserver; ipy_vimserver.setup('session'), and you can connect. * running the current file in IPython * running the current line / selection in IPython * running the current file / selection / line in IPython with the profiler enabled * growl notification of salient points * rudimentary syntax highlighting of ipythonrc files, including warnings about broken comments * commands for editing the ipythonrc file and .ipython directory * built in help * access from a single key binding. The bundle can be downloaded from [2] or installed via GetBundles. For installation help, see [2]. We'd love to hear your feedback, so please send comments, suggestions and feedback to either the ipython-dev list (please prefix the subject with [TextMate]), or the ipython-tmbundle google group (just reply to this email). Thanks! [1]: http://www.vimeo.com/2281439 [2]: http://github.com/mattfoster/ipython-tmbundle/ -- Matt Foster | http://hackerific.net --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "ipython-tmbundle" group. To post to this group, send email to ipython-tmbundle at googlegroups.com To unsubscribe from this group, send email to ipython-tmbundle+unsubscribe at googlegroups.com For more options, visit this group at http://groups.google.com/group/ipython-tmbundle?hl=en -~----------~----~----~----~------~----~------~--~--- From matt.p.foster at gmail.com Sat Nov 22 15:45:57 2008 From: matt.p.foster at gmail.com (Matt Foster) Date: Sat, 22 Nov 2008 20:45:57 +0000 Subject: [IPython-dev] [TextMate] IPython Bundle to E-texteditor github port In-Reply-To: <8C2B20C4348091499673D86BF10AB6762407BB0D@clipper.glosten.local> References: <1dcb3f200811181459m60d6df71k83f7edfbd98cf358@mail.gmail.com> <8C2B20C4348091499673D86BF10AB6762407BB0D@clipper.glosten.local> Message-ID: <1dcb3f200811221245q5a719ad6vef86723ff15a5484@mail.gmail.com> On Thu, Nov 20, 2008 at 10:51 PM, Benjamin J. Racine wrote: > Matt and all, > > http://github.com/gtcaz/ebundles/tree/master/Bundles > > Have you noticed that this is where things seem to get "ported" over to e-texteditor? > > I could be a tester from the e-texteditor side. Do you use the e-texteditor? I've seen it, but have no windows machines, so haven't used it. Anyway, I don't see any reason why we shouldn't try and keep things as platform independent as we can. Provided unix sockets work on windows, most of the functionality we have so far should be dead easy to port -- the main exception being the applescript stuff. Matt -- Matt Foster | http://hackerific.net From vishal.vatsa at gmail.com Mon Nov 24 09:56:08 2008 From: vishal.vatsa at gmail.com (Vishal Vatsa) Date: Mon, 24 Nov 2008 14:56:08 +0000 Subject: [IPython-dev] ipcluster with clusterfile References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130809031512k48c1cd7cn6632f8a26bb757e@mail.gmail.com> <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> <6ce0ac130811050919s1eae7fc8l9e8020ff4420c8db@mail.gmail.com> <6ce0ac130811161446y52aa965fx3394b0a40677a67c@mail.gmail.com> <6ce0ac130811191133q72bb2dc2wf03367d8c63879b@mail.gmail.com> Message-ID: Brian Granger wrote: >>> If you start hacking on the ssh cluster, keep in touch so we can >>> coordinate. Hi Brian, I have done my 1st drop of a working ssh cluster (sort of manager), http://bazaar.launchpad.net/%7Evvatsa/ipython/ipcluster-dev/revision/1165?start_revid=1165 Works in a similar way as to 0.8.x ipcluster, the main caveat is that this is very *nix, don't think this will work in windows, I have tested this on GNU/Linux systems, no reason it should not work on OS X. Have a look and if you have any pointers, let me know. I wanted to be happy with the cluster semantics 1st and then I would be happy to fix any implementation issues. Regards, -vishal From matt.p.foster at gmail.com Tue Nov 25 17:44:16 2008 From: matt.p.foster at gmail.com (Matt Foster) Date: Tue, 25 Nov 2008 22:44:16 +0000 Subject: [IPython-dev] ipy_editor.py: textmate function Message-ID: <1dcb3f200811251444g11c5cb45j97bf27ae715c78d7@mail.gmail.com> Hi, You can add TextMate support to ipy_editor.py like this (-w means wait): def mate(exe = 'mate'): """ TextMate, the missing editor""" install_editor(exe + '-w -l $line "$file"') I've attached a patch. Cheers, Matt -- Matt Foster | http://hackerific.net -------------- next part -------------- A non-text attachment was scrubbed... Name: ipy_editor_mate.diff Type: application/octet-stream Size: 441 bytes Desc: not available URL: From scott.p.macdonald at gmail.com Tue Nov 25 17:54:48 2008 From: scott.p.macdonald at gmail.com (Scott MacDonald) Date: Tue, 25 Nov 2008 15:54:48 -0700 Subject: [IPython-dev] ipy_editor.py: textmate function In-Reply-To: <1dcb3f200811251444g11c5cb45j97bf27ae715c78d7@mail.gmail.com> References: <1dcb3f200811251444g11c5cb45j97bf27ae715c78d7@mail.gmail.com> Message-ID: Nice work :) Now if only I could use mac's at work... On Tue, Nov 25, 2008 at 3:44 PM, Matt Foster wrote: > Hi, > > You can add TextMate support to ipy_editor.py like this (-w means wait): > > def mate(exe = 'mate'): > """ TextMate, the missing editor""" > install_editor(exe + '-w -l $line "$file"') > > I've attached a patch. > > Cheers, > > Matt > > -- > Matt Foster | http://hackerific.net > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjracine at glosten.com Tue Nov 25 19:00:38 2008 From: bjracine at glosten.com (Benjamin J. Racine) Date: Tue, 25 Nov 2008 16:00:38 -0800 Subject: [IPython-dev] ipy_editor.py: textmate function In-Reply-To: <1dcb3f200811251444g11c5cb45j97bf27ae715c78d7@mail.gmail.com> Message-ID: <8C2B20C4348091499673D86BF10AB676300CA5B4@clipper.glosten.local> Matt, This allows you to open an instance of textmate from ipython? On another note: I think I will commit some snippets soon. I like your unified approach, i.e. as few distinct keystrokes as possible, and leaving the rest to the pop-up menus, i.e. 'import tab' yields all the import statements at once... further 'plot tab' yields all the plotting statements at once. I might hold off a little longer on committing anything just yet, as I don't want to put anything out there until I feel that I've chosen the most rational shortcuts possible, which becomes clearer after a bit of effort. Right now I am trying to pull out any of the reoccuring material from EPD/Examples/Traits, EPD/Examples/TraitsGUI, EPD/Examples/Chaco, EPD/Examples/Mayavi Rgds, Ben R. -----Original Message----- From: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] On Behalf Of Matt Foster Sent: Tuesday, November 25, 2008 2:44 PM To: ipython-dev at scipy.net Subject: [IPython-dev] ipy_editor.py: textmate function Hi, You can add TextMate support to ipy_editor.py like this (-w means wait): def mate(exe = 'mate'): """ TextMate, the missing editor""" install_editor(exe + '-w -l $line "$file"') I've attached a patch. Cheers, Matt -- Matt Foster | http://hackerific.net From ellisonbg.net at gmail.com Tue Nov 25 19:50:06 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Tue, 25 Nov 2008 16:50:06 -0800 Subject: [IPython-dev] ipy_editor.py: textmate function In-Reply-To: <1dcb3f200811251444g11c5cb45j97bf27ae715c78d7@mail.gmail.com> References: <1dcb3f200811251444g11c5cb45j97bf27ae715c78d7@mail.gmail.com> Message-ID: <6ce0ac130811251650k11e093aetfc5c33f3e20dd5a9@mail.gmail.com> Matt, Thanks for this code, I am adding it right now. But when I tested it, it doesn't properly execute the code after editing. Did you get this to work? I saw a couple of different things: * New file If I do %edit newfile.py I get: In [15]: edit newfile.py Editing... done. Executing edited code... Could not open file for safe execution. This all happens _before_ I edit, save and close the file. It looks like -w is not working? * When a file already exists, the old, preedited version gets executed. Could you look into ways of fixing these things? Cheers, Brian IPython gives an error On Tue, Nov 25, 2008 at 2:44 PM, Matt Foster wrote: > Hi, > > You can add TextMate support to ipy_editor.py like this (-w means wait): > > def mate(exe = 'mate'): > """ TextMate, the missing editor""" > install_editor(exe + '-w -l $line "$file"') > > I've attached a patch. > > Cheers, > > Matt > > -- > Matt Foster | http://hackerific.net > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > > From ellisonbg.net at gmail.com Tue Nov 25 20:29:52 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Tue, 25 Nov 2008 17:29:52 -0800 Subject: [IPython-dev] ipcluster with clusterfile In-Reply-To: References: <6ce0ac130809031419g3bd07c7cxa1657c3f4f88b779@mail.gmail.com> <6ce0ac130811050851t13b5ff0ag1bcf84af3552c90e@mail.gmail.com> <6ce0ac130811050919s1eae7fc8l9e8020ff4420c8db@mail.gmail.com> <6ce0ac130811161446y52aa965fx3394b0a40677a67c@mail.gmail.com> <6ce0ac130811191133q72bb2dc2wf03367d8c63879b@mail.gmail.com> Message-ID: <6ce0ac130811251729o995bbfas3f42473c7e12a191@mail.gmail.com> Vishal, Cool! Can you create a launchpad proposal to merge (into my branch) for your branch. That way others can review your work as well and we can keep all the comments together. Once you do that I will begin looking at this. Cheers, Brian On Mon, Nov 24, 2008 at 6:56 AM, Vishal Vatsa wrote: > > Brian Granger wrote: > >>>> If you start hacking on the ssh cluster, keep in touch so we can >>>> coordinate. > Hi Brian, > > I have done my 1st drop of a working ssh cluster (sort of manager), > > http://bazaar.launchpad.net/%7Evvatsa/ipython/ipcluster-dev/revision/1165?start_revid=1165 > > Works in a similar way as to 0.8.x ipcluster, the main caveat is that this is very *nix, don't think this will work in windows, I have tested this on GNU/Linux systems, no reason it should not work on OS X. > > Have a look and if you have any pointers, let me know. > I wanted to be happy with the cluster semantics 1st and then I would be happy to fix any implementation issues. > > Regards, > -vishal > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Tue Nov 25 22:41:55 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Tue, 25 Nov 2008 19:41:55 -0800 Subject: [IPython-dev] ipy_render In-Reply-To: <1dcb3f200811180335s69acbbdbt770cd3c47e6fef4@mail.gmail.com> References: <1dcb3f200811170404v137ced58i247aba04949b5670@mail.gmail.com> <46cb515a0811171230p47988a0w97e5527c1646dac2@mail.gmail.com> <1dcb3f200811180225u40a6dd6bjec785440462d4107@mail.gmail.com> <1dcb3f200811180335s69acbbdbt770cd3c47e6fef4@mail.gmail.com> Message-ID: <6ce0ac130811251941w36159e53k9e55f48fd91afd1c@mail.gmail.com> Matt and Ville, What is the status of this? Once thing that might help in this case (as edits are done) is for Matt to create a new branch like: lp:~mfoster/ipython/ipython-ipyrender (or whatever your launchpad login is). That way all of can see exactly what the changes are in context. Then you can create (in launchpad) a "proposal to merge" for this branch and we can all review it. Let us know if you need help navigating launchpad (it can be a bit confusing at times). Thanks! Brian 2008/11/18 Matt Foster : > On Tue, Nov 18, 2008 at 10:25 AM, Matt Foster wrote: >> >> I'm happy to improve the submission and send it again. >> > > Hi Again, > > I know it hasn't been long since I sent the last email but I've been > playing around, and made a patch against lp:ipython. > > The main changes are: > * Added read_from_cmd and write_to_cmd to platutils_posix. > * Added check_cmd which uses read_from_cmd to run which and check > the output status. > * Added topclip functions for systems with pbcopy (mac) and xsel. > * Moved the win32 toclip stuff from ipy_render into platutils_win32. > * Finally, I modified the ipy_render file to use the new functions. > > I've tested this on my intel mac and linux boxes, but don't have any > windows machines handy, so I can't check that side of things. > > I'm happy to make any more changes you think might be needed, and > would appreciate any more feedback. > > See attached. > > Cheers, > > Matt > > -- > Matt Foster | http://hackerific.net > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > > From vivainio at gmail.com Wed Nov 26 01:52:08 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 26 Nov 2008 08:52:08 +0200 Subject: [IPython-dev] ipy_render In-Reply-To: <6ce0ac130811251941w36159e53k9e55f48fd91afd1c@mail.gmail.com> References: <1dcb3f200811170404v137ced58i247aba04949b5670@mail.gmail.com> <46cb515a0811171230p47988a0w97e5527c1646dac2@mail.gmail.com> <1dcb3f200811180225u40a6dd6bjec785440462d4107@mail.gmail.com> <1dcb3f200811180335s69acbbdbt770cd3c47e6fef4@mail.gmail.com> <6ce0ac130811251941w36159e53k9e55f48fd91afd1c@mail.gmail.com> Message-ID: <46cb515a0811252252x5d2342cdi83322d6ea3c77ccd@mail.gmail.com> On Wed, Nov 26, 2008 at 5:41 AM, Brian Granger wrote: > Matt and Ville, > > What is the status of this? Once thing that might help in this case > (as edits are done) is for Matt to create a new branch like: > > lp:~mfoster/ipython/ipython-ipyrender There is already a branch for this, it's basically ready to merge. https://code.launchpad.net/~villemvainio/ipython/ipyrender-linuxfix -- Ville M. Vainio http://tinyurl.com/vainio From matt.p.foster at gmail.com Wed Nov 26 03:38:23 2008 From: matt.p.foster at gmail.com (Matt Foster) Date: Wed, 26 Nov 2008 08:38:23 +0000 Subject: [IPython-dev] ipy_editor.py: textmate function In-Reply-To: <1dcb3f200811251444g11c5cb45j97bf27ae715c78d7@mail.gmail.com> References: <1dcb3f200811251444g11c5cb45j97bf27ae715c78d7@mail.gmail.com> Message-ID: <1dcb3f200811260038qd623adbt1578773187fa6370@mail.gmail.com> On Tue, Nov 25, 2008 at 10:44 PM, Matt Foster wrote: > Hi, > > You can add TextMate support to ipy_editor.py like this (-w means wait): > > def mate(exe = 'mate'): > """ TextMate, the missing editor""" > install_editor(exe + '-w -l $line "$file"') > > I've attached a patch. Arg. I think an extra space is needed before the '-w': def mate(exe = 'mate'): """ TextMate, the missing editor""" install_editor(exe + ' -w -l $line "$file"') Apologies for that. It's what I get for emailing a patch straight before bed. -- Matt Foster | http://hackerific.net -------------- next part -------------- A non-text attachment was scrubbed... Name: ipy_editor_mate.diff Type: application/octet-stream Size: 442 bytes Desc: not available URL: From matt.p.foster at gmail.com Wed Nov 26 03:45:59 2008 From: matt.p.foster at gmail.com (Matt Foster) Date: Wed, 26 Nov 2008 08:45:59 +0000 Subject: [IPython-dev] ipy_editor.py: textmate function In-Reply-To: <8C2B20C4348091499673D86BF10AB676300CA5B4@clipper.glosten.local> References: <1dcb3f200811251444g11c5cb45j97bf27ae715c78d7@mail.gmail.com> <8C2B20C4348091499673D86BF10AB676300CA5B4@clipper.glosten.local> Message-ID: <1dcb3f200811260045p6caf268bpb14371d22aafedb4@mail.gmail.com> On Wed, Nov 26, 2008 at 12:00 AM, Benjamin J. Racine wrote: > Matt, > > This allows you to open an instance of textmate from ipython? That's right. If you put the following into ~/.ipython/ipy_user.conf import ipy_editors ipy_editors.mate() Then it'll use it for the %edit command, and then execute the file after you've saved it and closed the window. > On another note: > > I think I will commit some snippets soon. I like your unified approach, i.e. as few distinct keystrokes as possible, and leaving the rest to the pop-up menus, i.e. > > 'import tab' yields all the import statements at once... further > 'plot tab' yields all the plotting statements at once. > > I might hold off a little longer on committing anything just yet, as I don't want to put anything out there until I feel that I've chosen the most rational shortcuts possible, which becomes clearer after a bit of effort. > > Right now I am trying to pull out any of the reoccuring material from > EPD/Examples/Traits, > EPD/Examples/TraitsGUI, > EPD/Examples/Chaco, > EPD/Examples/Mayavi > That sounds like it'll be cool. You can commit as much as you want into github, and as often as you want. Things like keybindings are trivial to change too, so commit when you're ready. Cheers, Matt -- Matt Foster | http://hackerific.net From ellisonbg.net at gmail.com Fri Nov 28 18:55:56 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 28 Nov 2008 15:55:56 -0800 Subject: [IPython-dev] New Twisted based server that works with IPython TextMate bundle Message-ID: <6ce0ac130811281555w64ea82bk26db3b410ee42f1a@mail.gmail.com> Hi, I just finished the first draft of a new Twisted based ipy_textmateserver.py extension that allows IPython to work with Matt Foster's IPython TextMate bundle. This replaces the stuff in ipy_vimserver.py and is much more powerful and flexible. To use it do: In [1]: import ipy_textmateserver In [2]: s = ipy_textmateserver.TextMateServer() In [3]: s.start() Then the bundle will find this session. Barry and other Mac+TextMate users, could you have a look at this to see what you think? Cheers, Brian