From Fernando.Perez at colorado.edu Mon Nov 8 16:10:16 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Mon, 08 Nov 2004 14:10:16 -0700 Subject: [IPython-dev] ANN: IPython 0.6.4 is out. Message-ID: <418FE0B8.1060502@colorado.edu> Hi all, I'm glad to announce that IPython 0.6.4 is finally out. You can download it from the usual place: http://ipython.scipy.org/dist/ I've provided RPMs for Python 2.2 and 2.3, plus source downloads (.tar.gz and .zip). Debian, Fink and BSD packages for this version should be coming soon, as the respective maintainers (many thanks to Jack Moffit, Andrea Riciputi and Dryice Liu) have the time to follow their packaging procedures. Many thanks to Enthought for their continued hosting support for IPython, and to all the users who contributed ideas, fixes and reports. IMPORTANT NOTE: Unless any important bugs are found, this version will likely be the last ipython release for quite some time. I intend to (finally) start a major internal cleanup, as described in: http://ipython.scipy.org/doc/new_design.pdf This process will likely take a while; even though it's relatively straightforward, there is a _lot_ of code to go through. In a few weeks, I will probably open a separate CVS branch for this, so that I can still fix bugs for version 0.6.4 in CVS HEAD if they are found. Needless to say, any volunteers are welcome to help :) Once the new branch becomes reasonably stable, it will be released, and I hope this will soon after lead to an official '1.0' release for ipython. Now, on to the usual summary of changes in IPython 0.6.4 vs 0.6.3. As always, the complete NEWS file can be found at http://ipython.scipy.org/NEWS, and the whole ChangeLog at http://ipython.scipy.org/ChangeLog. - License changed to BSD: IPython is now fully licensed under the terms of the BSD license (except for a few MIT-licensed files which have always been there; the MIT license is very similar to BSD). - Changed the explicit magic escape from @ to %, to conform with the introduction of Python @decorators in Python 2.4. IPython 0.6.4 should be 100% compatible with Python 2.4. - Many small improvements which mainly matter to users of the matplotlib support (via -pylab). These fix a number of problems, crashes and annoyances present in 0.6.3. - Small fixes and hardenings against crashes which could be caused by buggy third-party code run from within ipython. There are currently no known ipython crash cases. Enjoy, and as usual please report any problems. Regards, Fernando. From smirk at thebuicksix.com Wed Nov 10 02:36:15 2004 From: smirk at thebuicksix.com (Donny Viszneki) Date: Wed, 10 Nov 2004 02:36:15 -0500 Subject: [IPython-dev] Code Indentation and Tab Completion Message-ID: <33C8F0E6-32EB-11D9-9DB7-000A95D342AC@thebuicksix.com> I'm using Mac OS X, and in every terminal I use, when I press tab to indent for a block of code, iPython instead tries to tab complete, and consequently asks me if I really want to see every possible tab completion. What's the problem here? I have the source code so I was thinking about taking a look. Does iPython use curses? Thanks in advance. From ariciputi at pito.com Wed Nov 10 03:38:31 2004 From: ariciputi at pito.com (Andrea Riciputi) Date: Wed, 10 Nov 2004 09:38:31 +0100 Subject: [IPython-dev] Code Indentation and Tab Completion In-Reply-To: <33C8F0E6-32EB-11D9-9DB7-000A95D342AC@thebuicksix.com> References: <33C8F0E6-32EB-11D9-9DB7-000A95D342AC@thebuicksix.com> Message-ID: Hi, under IPython the tab key is bounded to auto-completion so I think it's hard to bound it to indentation too and leave IPython to decide which function you want when you press that key. Perhaps you can make IPython aware of tab at the beginning of a line or something like that, but at that point you will end up with a key bounded to two different actions and I don't know if it is advisable. But Fernando can give us more insight in this. At the moment I solve this problem by typing a space (instead of a tab) when I want to indent a code block. HTH, Andrea. On 10 Nov 2004, at 08:36, Donny Viszneki wrote: > I'm using Mac OS X, and in every terminal I use, when I press tab to > indent for a block of code, iPython instead tries to tab complete, and > consequently asks me if I really want to see every possible tab > completion. > > What's the problem here? I have the source code so I was thinking > about taking a look. Does iPython use curses? > > Thanks in advance. From Fernando.Perez at colorado.edu Wed Nov 10 02:43:50 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Wed, 10 Nov 2004 00:43:50 -0700 Subject: [IPython-dev] Code Indentation and Tab Completion In-Reply-To: <33C8F0E6-32EB-11D9-9DB7-000A95D342AC@thebuicksix.com> References: <33C8F0E6-32EB-11D9-9DB7-000A95D342AC@thebuicksix.com> Message-ID: <4191C6B6.8040201@colorado.edu> Donny Viszneki wrote: > I'm using Mac OS X, and in every terminal I use, when I press tab to > indent for a block of code, iPython instead tries to tab complete, and > consequently asks me if I really want to see every possible tab > completion. > > What's the problem here? I have the source code so I was thinking about > taking a look. Does iPython use curses? It's a feature :) No, ipython doesn't use curses, but it uses GNU readline and configures it for tab-completion. Most users (myself included) prefer the tab key to do completions rather than indentation when working interactively. But like all things ipython, you can change this to suit your personal taste. Look at the ~/.ipython/ipythonrc file for all of ipython's configuration options, in particular see the readline section. If you simply comment out the readline_parse_and_bind tab: line, the tab key will be left alone. Note that ipython, as it rebinds tab for completions, also rebinds the following: readline_parse_and_bind "\C-o": tab-insert readline_parse_and_bind "\M-i": " " readline_parse_and_bind "\M-o": "\d\d\d\d" readline_parse_and_bind "\M-I": "\d\d\d\d" (the \d means backspace in readline parlance). This hopefully gives you enough options for a normal workflow, keeping the power of tab-completion active (which most of us simply can't live without). Best, f From Fernando.Perez at colorado.edu Wed Nov 10 04:02:23 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Wed, 10 Nov 2004 02:02:23 -0700 Subject: [IPython-dev] Code Indentation and Tab Completion In-Reply-To: <76DF7284-32F6-11D9-9DB7-000A95D342AC@thebuicksix.com> References: <33C8F0E6-32EB-11D9-9DB7-000A95D342AC@thebuicksix.com> <4191C6B6.8040201@colorado.edu> <76DF7284-32F6-11D9-9DB7-000A95D342AC@thebuicksix.com> Message-ID: <4191D91F.3080600@colorado.edu> Donny Viszneki wrote: > Here's a theory: > > What if to the tab-completion list we were to add a one character > string that just consisted of Tab? Would this hack be sufficient to > allow me to press tab on a blank line for the proper indentation? You'd still have the problem of having readline asking you about 634 possible completions on any blank line (which would include your hypothetical tab entry). So I don't see how this would solve anything: if you have to hit tab, then 'yes', then tab again, might as well just type one or two spaces (or use Ctrl-O, or whatever you want to rebind in readline as I indicated). Note that hitting tab to indent an existing line where you've already started typing, like for example Xemacs can do, is simply beyond the capabilities of readline. So the only place for tab to work as you'd want is at the very beginning, while the line is still blank (and hence open to the problem of a zillion alternate completions). And even if you wanted to do this, I doubt it could work at all, as I think that readline simply traps the tab key event immediately, and you could never 'complete' the fake entry you suggest. But I don't have the time to test this hack (which I dislike on technical grounds, and suspect will fail). If you find that it works, post a patch on the list and I'll gladly have a look at it. Best, f From smirk at thebuicksix.com Wed Nov 10 04:14:20 2004 From: smirk at thebuicksix.com (Donny Viszneki) Date: Wed, 10 Nov 2004 04:14:20 -0500 Subject: [IPython-dev] iPython, Mac OS X, Preview.app, and GraphViz Message-ID: One of the best features of the python shell, and iPython, is the online help. I was thinking, how hard would it be to create an alternate online help system that wrote a DOT (google for GraphViz) file showing hierarchically all the members (and their members, perhaps) of the object or class passed to this hypothetical help funciton. With some more work, the DOT file could be converted into the first page of a PDF, and perhaps all the objects shown in the picture generated by GraphViz could act as "links" (I don't know what term is used in PDFs, but in many PDFs plenty of "links" are clickable just like an HTML document.) And these links could go to a page in the PDF with the ordinary printout of the online help. This would be really useful for someone on Mac OS X. If anyone else is interested on working on something like this let me know. From Fernando.Perez at colorado.edu Wed Nov 10 04:15:31 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Wed, 10 Nov 2004 02:15:31 -0700 Subject: [IPython-dev] iPython, Mac OS X, Preview.app, and GraphViz In-Reply-To: References: Message-ID: <4191DC33.1000302@colorado.edu> Donny Viszneki wrote: > One of the best features of the python shell, and iPython, is the > online help. > > I was thinking, how hard would it be to create an alternate online help > system that wrote a DOT (google for GraphViz) file showing > hierarchically all the members (and their members, perhaps) of the > object or class passed to this hypothetical help funciton. > > With some more work, the DOT file could be converted into the first > page of a PDF, and perhaps all the objects shown in the picture > generated by GraphViz could act as "links" (I don't know what term is > used in PDFs, but in many PDFs plenty of "links" are clickable just > like an HTML document.) And these links could go to a page in the PDF > with the ordinary printout of the online help. This might be a useful starting point, as it seems to cover at least part of what you want: http://www.tarind.com/depgraph.html. It's a dependency tree instead of a member one, but probably much of the code would be reusable. best, f From rkern at ucsd.edu Wed Nov 10 04:29:55 2004 From: rkern at ucsd.edu (Robert Kern) Date: Wed, 10 Nov 2004 01:29:55 -0800 Subject: [IPython-dev] Re: Code Indentation and Tab Completion In-Reply-To: <33C8F0E6-32EB-11D9-9DB7-000A95D342AC@thebuicksix.com> References: <33C8F0E6-32EB-11D9-9DB7-000A95D342AC@thebuicksix.com> Message-ID: Donny Viszneki wrote: > I'm using Mac OS X, and in every terminal I use, when I press tab to > indent for a block of code, iPython instead tries to tab complete, and > consequently asks me if I really want to see every possible tab completion. > > What's the problem here? I have the source code so I was thinking about > taking a look. Does iPython use curses? > > Thanks in advance. In addition to what the others said, also take a look at the autoindent feature (toggled via %autoindent or set in your ipythonrc). I never indent anymore, just dedent. -- Robert Kern rkern at ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter From prabhu_r at users.sf.net Sun Nov 14 14:25:34 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Mon, 15 Nov 2004 00:55:34 +0530 Subject: [IPython-dev] Generic gui_thread + IPython: solution already exists! Message-ID: <16791.45358.186631.80855@monster.linux.in> Hi, [Sorry about the cross-posting, but I think this is relevant on both lists.] I just happened to read Fernando's post on c.l.py on the way in which IPython manages to get gtk and wxPython working for matplotlib. Here is the thread: http://xrl.us/dyej I then looked at IPython/Shell.py and realized that IPython can actually be trivially extended to work with *any* wx or gtk app! I actually thought of using this approach earlier when I was fixing gui_thread, but thought IPython would need too many modifications for this to work. However, it looks like Fernando has already done all the work for us. wxPython -------- If you just start ipython -pylab you can run the wxPythonDemo completely interactively from IPython:: In [1]: cd /usr/local/share/doc/wxPython-2.5.2.8/demo/ /usr/local/share/doc/wxPython-2.5.2.8/demo In [2]: import Main In [3]: d = Main.wxPythonDemo(None, 'a') In [4]: d.Show(1) Out[4]: True In [5]: This works out nicely! I also tried a bunch of other tests and things work well. Now, obviously, no other application can launch a mainloop. So we need to somehow inhibit the user from being able to start it. The solution is incredibly simple, replace the real mainloop with a dummy. Here is a simple example that runs the wxVTKRenderWindowInteractor example from within IPython! I've just cut/pasted this from IPython's history:: # This is with wxPython-2.5.x; We can do something very similar for # wxPython-2.4.x. ########################################### import wx def Dummy_Mainloop(*args, **kw): pass wx._core_.PyApp_MainLoop = Dummy_Mainloop ########################################### from vtk.wx import wxVTKRenderWindowInteractor wxVTKRenderWindowInteractor.wxVTKRenderWindowInteractorConeExample() # Now we see our familiar cone, but the interpreter is still active! wxVTKRenderWindowInteractor.wxVTKRenderWindowInteractorConeExample() # One more cone! No, there is no gui_thread in use here! So, the theory works out in practice. Thus with a trivial addition to IPython, almost *ANY* wxPython app can be made to work perfectly interactively! GTK --- If you get the drift of what I did to get wxPython working interactively for arbitrary apps, you can see where this is headed. So lets try this approach for gtk. In [1]: import gtk In [2]: def f(): ...: pass ...: In [3]: gtk.main = f In [4]: import menu In [5]: m = menu.MenuExample() # The example pops up at this point and is fully useable. In [6]: gtk.main() [...] In [8]: menu.main() Out[8]: 0 The menu.py example is part of the python-gtk2-tutorial package on Debian Sarge. Obviously, the trivial example in the python-gtk2-tutorial also works just fine. You don't really need to hijack main, but it helps. Now, I'm *really* rusty with pyGTK and I would think you'd need to hijack a few more functions for this to work perfectly (gtk.mainloop?). However the changes are minimal. Unlike the current gui_thread approach, which takes forever to startup, the IPython approach is fast since we are only hijacking one or two functions and not the whole wxPython API! So I think these additions would make IPython a reliable way to interactively run most wxPython and GTK apps. I also vote that once these changes are made, -gthread and -wthread options be made available. cheers, prabhu From Fernando.Perez at colorado.edu Sun Nov 14 16:54:20 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Sun, 14 Nov 2004 14:54:20 -0700 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16791.45358.186631.80855@monster.linux.in> References: <16791.45358.186631.80855@monster.linux.in> Message-ID: <4197D40C.6060103@colorado.edu> Prabhu Ramachandran wrote: > [Sorry about the cross-posting, but I think this is relevant on both > lists.] > > I just happened to read Fernando's post on c.l.py on the way in which > IPython manages to get gtk and wxPython working for matplotlib. Here > is the thread: > > http://xrl.us/dyej > > I then looked at IPython/Shell.py and realized that IPython can > actually be trivially extended to work with *any* wx or gtk app! > > I actually thought of using this approach earlier when I was fixing > gui_thread, but thought IPython would need too many modifications for > this to work. However, it looks like Fernando has already done all > the work for us. Well, I think the real credit should go to John Hunter and Antoon Pardon, the people who cracked the nut of the threading code originally :) This is great news, obviously! This is very much what Eric and I originally discussed at scipy'04, and what I thought might be possible to do. While fumbling in the dark with my near-zero knowledge of threading, in an attempt to get matplolib running, I finally thought that perhaps this approach was just not viable, and that the full complexity of gui_thread was indeed inevitable. I basically was not willing to believe that such a simple approach could indeed substitute all the functionality of gui_thread, and gave up in my ignorance. I'm happy to see that my original hunch was indeed right :) The leftover machinery for -wthread/-gthread which you see there is all that's left of my efforts. I'm glad I didn't completely wipe that out. Re-enabling it is trivial, and I even had written the relevant manual parts for it, so this would be an easy fix. I would be more than happy to put out a 0.6.5 release with these fixes, since I think they would be a _major_ enhancement to ipython. People have been asking for a long time for a way to use wx/gtk interactively, and if this solution truly can substitute gui_thread, I think it's a major win. gui_thread is complex and brittle, and the work of maintaing it is a solid time sink with a developer bottleneck: basically only you and Pearu (I think) understand that code enough to dare stick your fingers in it. IMO it would be great if we could just get rid of it without loss of functionality. I don't have the necessary expertise to properly test this out for both wx/gtk, but I'd love to see these improvements go in, so I hope you or someone else can finish them up. Great work! Best, f From prabhu_r at users.sf.net Sun Nov 14 21:26:33 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Mon, 15 Nov 2004 07:56:33 +0530 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: References: <16791.45358.186631.80855@monster.linux.in> Message-ID: <16792.5081.179029.452649@monster.linux.in> >>>>> "PP" == Pearu Peterson writes: PP> On Mon, 15 Nov 2004, Prabhu Ramachandran wrote: PP> With wx 2.4.2.4 I get: PP> In [1]: cd /opt/src/wxPythonSrc-2.4.2.4/wxPython/demo PP> /opt/src/wxPythonSrc-2.4.2.4/wxPython/demo PP> In [2]: import Main PP> In [3]: d = Main.wxPythonDemo(None, -1, 'a') 22:39:45: [...] PP> AssertionError: No wxApp created yet Looks like you did not start IPython with the -pylab option? wxApp is created if you use -pylab (AND set WX as your default backend in .matplotlibrc, I forgot to add this detail). Mine looks like this:: backend : WX interactive : True Put this in a file called ~/.matplotlibrc and start IPython like so:: $ ipython -pylab then try again. Thanks! >> Now, obviously, no other application can launch a mainloop. So >> we need to somehow inhibit the user from being able to start >> it. The solution is incredibly simple, replace the real >> mainloop with a dummy. PP> Could this be used in gui_thread? Do we need gui_thread PP> anymore? gui_thread can hang around FWIW. The solution I am discussion will only work with IPython. Not with vanilla Python or with IDLE etc. I don't use these so this is not an issue but I think gui_thread can be left as it is (with some simple cleanup perhaps). This is one more reason why IPython should be the default Python shell for anyone who uses Python interactively. :) The approach Fernando uses is identical to what is done in the gpython.py shell used to run pyGTK interactively. A wxApp or gtk.mainloop() is run in a background thread and *everything* typed by the user is run in it. IPython grabs whatever the user types and then sends this off to the second thread and executes it there. Thus, things are always done in the right thread and there are no issues. The only problem is exceptions, signal handlers etc. It looks like exceptions are trapped nicely and passed back to the main thread. I don't know about the signal handlers. cheers, prabhu From Fernando.Perez at colorado.edu Sun Nov 14 21:32:43 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Sun, 14 Nov 2004 19:32:43 -0700 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16792.5081.179029.452649@monster.linux.in> References: <16791.45358.186631.80855@monster.linux.in> <16792.5081.179029.452649@monster.linux.in> Message-ID: <4198154B.6020509@colorado.edu> Prabhu Ramachandran wrote: > The approach Fernando uses is identical to what is done in the > gpython.py shell used to run pyGTK interactively. A wxApp or > gtk.mainloop() is run in a background thread and *everything* typed by > the user is run in it. IPython grabs whatever the user types and then > sends this off to the second thread and executes it there. Thus, > things are always done in the right thread and there are no issues. > The only problem is exceptions, signal handlers etc. It looks like > exceptions are trapped nicely and passed back to the main thread. I > don't know about the signal handlers. Signal handling is a mess. The thread you referred to originally mentions this. My brute-force approach was to install a sigint handler _every_ time I enter the 2nd thread, right before executing user code. Somehow this seems to allow me to trap SIGINT correctly, which I otherwise could never catch (and it would blow things up badly). From what others on c.l.py said, this is not an easy problem, but if anyone knows of a cleaner solution than what I came up with (an admittedly gross hack), I'm all ears. Best, f From prabhu_r at users.sf.net Sun Nov 14 21:39:44 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Mon, 15 Nov 2004 08:09:44 +0530 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <4197D40C.6060103@colorado.edu> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> Message-ID: <16792.5872.782287.158081@monster.linux.in> >>>>> "FP" == Fernando Perez writes: >> I actually thought of using this approach earlier when I was >> fixing gui_thread, but thought IPython would need too many >> modifications for this to work. However, it looks like >> Fernando has already done all the work for us. FP> Well, I think the real credit should go to John Hunter and FP> Antoon Pardon, the people who cracked the nut of the threading FP> code originally :) Well, with all due respect to John, from Shell.py: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65109, by Brian McErlean and John Finlay. Modified with corrections by Antoon Pardon. So I think these guys should get the credit for the original idea. The good thing with IPython is that you control the prompt. So, this approach is workable for you. It won't work on vanilla Python or IDLE. That is more than good enough for me. :) FP> This is great news, obviously! Yes! FP> This is very much what Eric and I originally discussed at FP> scipy'04, and what I thought might be possible to do. While Though I was right in the middle of that conversation, I did not realize that you had taken the approach of pygtkconsole.py. If only I had known that, I would not have spent the time hacking gui_thread recently. Well, maybe that is a good thing because we have a partially working gui_thread if someone needs it. FP> fumbling in the dark with my near-zero knowledge of threading, FP> in an attempt to get matplolib running, I finally thought that FP> perhaps this approach was just not viable, and that the full FP> complexity of gui_thread was indeed inevitable. I basically FP> was not willing to believe that such a simple approach could FP> indeed substitute all the functionality of gui_thread, and FP> gave up in my ignorance. I'm happy to see that my original FP> hunch was indeed right :) :) FP> The leftover machinery for -wthread/-gthread which you see FP> there is all that's left of my efforts. I'm glad I didn't FP> completely wipe that out. Re-enabling it is trivial, and I FP> even had written the relevant manual parts for it, so this FP> would be an easy fix. Yes. FP> I would be more than happy to put out a 0.6.5 release with FP> these fixes, since I think they would be a _major_ enhancement FP> to ipython. Very good! FP> People have been asking for a long time for a way to use FP> wx/gtk interactively, and if this solution truly can FP> substitute gui_thread, I think it's a major win. gui_thread FP> is complex and brittle, and the work of maintaing it is a FP> solid time sink with a developer bottleneck: basically only FP> you and Pearu (I think) understand that code enough to dare FP> stick your fingers in it. IMO it would be great if we could FP> just get rid of it without loss of functionality. No need to get rid of it. The code can be used if someone needs it but the recommended approach will be to use IPython. FP> I don't have the necessary expertise to properly test this out FP> for both wx/gtk, but I'd love to see these improvements go in, FP> so I hope you or someone else can finish them up. I can send you a patch for the wxPython stuff and the preliminary work for gtk. I think John will have to fix the pyGTK code a little more since he obviously knows a lot more than me about pyGTK. FP> Great work! No, once again, I see that I'm just a blithering idiot. I should have seen this a long while ago. My ignorance has gotten the better of me again. Better late than never though. ;-) cheers, prabhu From jdhunter at ace.bsd.uchicago.edu Sun Nov 14 23:14:23 2004 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Sun, 14 Nov 2004 22:14:23 -0600 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16792.5081.179029.452649@monster.linux.in> (Prabhu Ramachandran's message of "Mon, 15 Nov 2004 07:56:33 +0530") References: <16791.45358.186631.80855@monster.linux.in> <16792.5081.179029.452649@monster.linux.in> Message-ID: >>>>> "Prabhu" == Prabhu Ramachandran writes: Prabhu> Looks like you did not start IPython with the -pylab Prabhu> option? wxApp is created if you use -pylab (AND set WX as Prabhu> your default backend in .matplotlibrc, I forgot to add Prabhu> this detail). Mine looks like this:: Prabhu> backend : WX interactive : True Prabhu> Put this in a file called ~/.matplotlibrc and start Prabhu> IPython like so:: I know you are using this just as a way of using ipython to control wx arbitrary apps, but just so people who might also be using or considering using matplotlib don't get the wrong idea, I suggest setting the backend to WXAgg rather than WX. Then you'll get all the wx stuff you need from ipython and in the event you generate any matplotlib plots, they'll be feature complete and look nice. The plain-vanilla wx backend lacks many core matplotlib features and I'm in no hurry to add them because I encourage people to use one of the *Agg backends whenever possible (like now). And now back to your regularly scheduled threading discussion.... JDH From Fernando.Perez at colorado.edu Mon Nov 15 00:37:13 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Sun, 14 Nov 2004 22:37:13 -0700 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16792.5872.782287.158081@monster.linux.in> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> Message-ID: <41984089.1070005@colorado.edu> Prabhu Ramachandran wrote: >>>>>>"FP" == Fernando Perez writes: > > > >> I actually thought of using this approach earlier when I was > >> fixing gui_thread, but thought IPython would need too many > >> modifications for this to work. However, it looks like > >> Fernando has already done all the work for us. > > FP> Well, I think the real credit should go to John Hunter and > FP> Antoon Pardon, the people who cracked the nut of the threading > FP> code originally :) > > Well, with all due respect to John, from Shell.py: > > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65109, > > by Brian McErlean and John Finlay. > Modified with corrections by Antoon Pardon. > So I think these guys should get the credit for the original idea. Yes, that's true. They wrote the original ASPN recipe with the basic threading trick. But I ran into a few nasty problems along the way (after rewriting that ASPN code completely, just keeping the core trick), and Antoon and John were the ones who fixed my hangups (and John got all the WX Timer stuff working). All in all, the best of open source, collaborative effort: none of us was an expert at this, but bit by bit we pieced together a working solution. > FP> I don't have the necessary expertise to properly test this out > FP> for both wx/gtk, but I'd love to see these improvements go in, > FP> so I hope you or someone else can finish them up. > > I can send you a patch for the wxPython stuff and the preliminary work > for gtk. I think John will have to fix the pyGTK code a little more > since he obviously knows a lot more than me about pyGTK. Sure, pass me the patch. Hopefully John, or someone else, can take a stab at this at so the pygtk stuff works as well. I'll give it a try myself, but since I've never used pygtk, I'm quite likely to miss even obvious stuff.> > FP> Great work! > > No, once again, I see that I'm just a blithering idiot. I should have > seen this a long while ago. My ignorance has gotten the better of me > again. Better late than never though. ;-) :) Anyway, this will be _very_ good to have in working order, and I think it will be something that many people even beyond the scipy crowd will benefit from. Best, f From prabhu_r at users.sf.net Mon Nov 15 05:33:08 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Mon, 15 Nov 2004 16:03:08 +0530 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <41984089.1070005@colorado.edu> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> Message-ID: <16792.34276.631159.477706@monster.linux.in> Hi Folks, >>>>> "FP" == Fernando Perez writes: [...] >> I can send you a patch for the wxPython stuff and the >> preliminary work for gtk. I think John will have to fix the >> pyGTK code a little more since he obviously knows a lot more >> than me about pyGTK. FP> Sure, pass me the patch. Hopefully John, or someone else, can FP> take a stab at this at so the pygtk stuff works as well. I'll FP> give it a try myself, but since I've never used pygtk, I'm FP> quite likely to miss even obvious stuff.> Attached is a patch for IPython/Shell.py. The diff is against the CVS version of Shell.py. A summary of what I've done: * Enabled the -wthread (for wx) and -gthread (for gtk) options. This means you don't need to do -pylab and setup the .matplotlibrc if you don't need it. * Hijack all relevant mainloops that I am aware of for wx and gtk. The wxPython support should work for both 2.4.x and 2.5.x. My simple tests for gtk also work just fine. So I think we have gtk covered as well. * BONUS: I also added support so you can script Tkinter apps when you use either -wthread or -gthread! This necessitates importing Tkinter at startup time, but that takes so little time, that I thought it not worth adding yet another command line option. Anyway, what this means is that you can actually use matplotlib/wxPython/pygtk apps *and* MayaVi at the *same* time. It all seems to work really nicely. :) Issues ------ It is NOT possible to run both wxPython and pyGTK apps together. Doing so will leave you with serious problems (most likely segfaults and core files). Tkinter works well with either. While this might not be a big deal (it certainly is not a big deal for me), it would be nice if this were handled. I have a crude idea to work around this by simply creating two threads, one for wx and one for gtk. We have two possibilities: 1. The user has to request that the command be executed in the appropriate thread. Something like this:: >>> %wx_thread Message: WX thread is default >>> d.Show(1) >>> %gtk_thread: w.show_all() >>> d.Show(0) ... >>> %gtk_thread Message: GTK thread is default >>> %wx_thread: d.Show(1) ... 2. When interpreting the code, check the types of the objects being used and send code to appropriate thread. I would think this is a painful way, plus you can't handle cases where both gtk and wx objects are used. Option 1 allows the user to blow up the Python session to bits if he does the wrong thing. However, it makes it possible to run GTK, WX and Tkinter apps interactively! Definitely worth bragging about. ;-) In any case, the attached patch is as far as I am willing to go. I currently don't need to run wxPython and gtk apps simultaneously and am unlikely to run into this situation in the near future. So, I'll let you think about the other issues. Enjoy! BTW, Fernando, did you get to post the modified python-mode.el to the python-mode maintainers? cheers, prabhu -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ipython.patch URL: From jdhunter at ace.bsd.uchicago.edu Mon Nov 15 10:35:15 2004 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Mon, 15 Nov 2004 09:35:15 -0600 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16792.34276.631159.477706@monster.linux.in> (Prabhu Ramachandran's message of "Mon, 15 Nov 2004 16:03:08 +0530") References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> Message-ID: >>>>> "Prabhu" == Prabhu Ramachandran writes: Prabhu> Attached is a patch for IPython/Shell.py. The diff is Prabhu> against the CVS version of Shell.py. A summary of what Prabhu> I've done: I haven't looked closely at this, but with ipython cvs + your patch, I'm getting a lot of ipython shell freezes in pylab mode with GTKAgg backend. JDH From prabhu_r at users.sf.net Mon Nov 15 10:56:24 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Mon, 15 Nov 2004 21:26:24 +0530 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> Message-ID: <16792.53672.53562.888095@monster.linux.in> >>>>> "JH" == John Hunter writes: >>>>> "Prabhu" == Prabhu Ramachandran writes: Prabhu> Attached is a patch for IPython/Shell.py. The diff is Prabhu> against the CVS version of Shell.py. A summary of what Prabhu> I've done: JH> I haven't looked closely at this, but with ipython cvs + your JH> patch, I'm getting a lot of ipython shell freezes in pylab JH> mode with GTKAgg backend. More information would be appreciated as would test cases. My simple tests seem to work ok. How about the other backends? WXAgg, WX, GTK and GTKAgg all seem to work ok for me. However, I have not really stressed anything out to really know for sure. BTW, I am using matplotlib version 0.62.4. cheers, prabhu From prabhu_r at users.sf.net Mon Nov 15 11:07:53 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Mon, 15 Nov 2004 21:37:53 +0530 Subject: [IPython-dev] Bug in patch for wxPython-2.4.x. Message-ID: <16792.54361.915823.81433@monster.linux.in> Hi, I made a cut/paste error for the wxPython-2.4.x support. Attached is a modified patch that should hopefully work. cheers, prabhu -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ipython.patch URL: From jdhunter at ace.bsd.uchicago.edu Mon Nov 15 11:04:56 2004 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Mon, 15 Nov 2004 10:04:56 -0600 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16792.53672.53562.888095@monster.linux.in> (Prabhu Ramachandran's message of "Mon, 15 Nov 2004 21:26:24 +0530") References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> Message-ID: >>>>> "Prabhu" == Prabhu Ramachandran writes: >>>>> "JH" == John Hunter writes: >>>>> "Prabhu" == Prabhu Ramachandran writes: Prabhu> Attached is a patch for IPython/Shell.py. The diff is Prabhu> against the CVS version of Shell.py. A summary of what Prabhu> I've done: JH> I haven't looked closely at this, but with ipython cvs + your JH> patch, I'm getting a lot of ipython shell freezes in pylab JH> mode with GTKAgg backend. Prabhu> More information would be appreciated as would test cases. Prabhu> My simple tests seem to work ok. How about the other Prabhu> backends? WXAgg, WX, GTK and GTKAgg all seem to work ok Prabhu> for me. However, I have not really stressed anything out Prabhu> to really know for sure. Prabhu> BTW, I am using matplotlib version 0.62.4. Oops, sorry, I'm lame. This happens in --gthread and --wthread and so doesn't appear to be matplotlib related; but for the record I'm using cvs matplotlib. The freeze can be triggered simply by typing anything at the prompt > ipython --gthread >>> x = FREEZE Ie, I just start typing some stuff, no additional imports, and the shell freezes. I can recover the shell by doing CTRL-C ENTER. It doesn't always happen, but it happens a lot. For example, in the session below, I tried to type x = 123 but the shell froze peds-pc311:~> ipython --wthread Python 2.3.3 (#2, Apr 13 2004, 17:41:29) Type "copyright", "credits" or "license" for more information. IPython 0.6.4 -- An enhanced Interactive Python. ? -> Introduction to IPython's features. %magic -> Information about IPython's 'magic' % functions. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. 1 >>> x = 12 KeyboardInterrupt - Press to continue. 2 >>> FYI, I had to hack hijack_wx because on my wx, there is no wx._core_. It appears to be wx._core. I don't know if a similar hack needs to be applied to the 2.4 section. 7 >>> wxPython.__version__ Out[7]: '2.5.1.5u' Here's the modified hijack def hijack_wx(): """Modifies wxPython's MainLoop with a dummy so user code does not block IPython. The hijacked mainloop function is returned. """ def dummy_mainloop(*args, **kw): pass import wxPython ver = wxPython.__version__ orig_mainloop = None if ver[:3] == '2.5': import wx if hasattr(wx, '_core_'): core = getattr(wx, '_core_') elif hasattr(wx, '_core'): core = getattr(wx, '_core') else: raise AttributeError('Could not find wx core module') orig_mainloop = core.PyApp_MainLoop core.PyApp_MainLoop = dummy_mainloop elif ver[:3] == '2.4': orig_mainloop = wx._core_.PyApp_MainLoop wxPython.wxc.wxPyApp_MainLoop = dummy_mainloop else: warn("Unable to find either wxPython version 2.4 or 2.5.") return orig_mainloop From prabhu_r at users.sf.net Mon Nov 15 11:40:04 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Mon, 15 Nov 2004 22:10:04 +0530 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> Message-ID: <16792.56292.811530.847847@monster.linux.in> >>>>> "JH" == John Hunter writes: >>>>> "Prabhu" == Prabhu Ramachandran writes: JH> Oops, sorry, I'm lame. JH> This happens in --gthread and --wthread and so doesn't appear JH> to be matplotlib related; but for the record I'm using cvs JH> matplotlib. Hmm, I did not know that --gthread also worked, I thought only -gthread and -wthread worked. The gthread option is really not doing anything fancy at all. All I've added is a function that updates Tkinter when the timer is called and removed gtk's own mainloop with a hijacked version. [...] JH> FYI, I had to hack hijack_wx because on my wx, there is no JH> wx._core_. It appears to be wx._core. I don't know if a JH> similar hack needs to be applied to the 2.4 section. wxPython-2.4 is I think is more reliable and should work out fine. You'll need my updated patch that I just posted though (it just changes one line). To improve the hijacking can you just check these out and let me know? Does wx._core have a _core_ attribute? If so, then you really need to access wx._core._core_ and hijack that, this works for me also, so if this works for you, I'll make that the default and we won't need any of the hasattr checks. JH> def hijack_wx(): [...] Does this hijacked hijack_wx work fine or does it still freeze? Thanks! cheers, prabhu From jdhunter at ace.bsd.uchicago.edu Mon Nov 15 12:27:51 2004 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Mon, 15 Nov 2004 11:27:51 -0600 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16792.56292.811530.847847@monster.linux.in> (Prabhu Ramachandran's message of "Mon, 15 Nov 2004 22:10:04 +0530") References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> Message-ID: >>>>> "Prabhu" == Prabhu Ramachandran writes: >>>>> "JH" == John Hunter writes: >>>>> "Prabhu" == Prabhu Ramachandran writes: JH> Oops, sorry, I'm lame. JH> This happens in --gthread and --wthread and so doesn't appear JH> to be matplotlib related; but for the record I'm using cvs JH> matplotlib. Prabhu> Hmm, I did not know that --gthread also worked, I thought Prabhu> only -gthread and -wthread worked. The gthread option is Prabhu> really not doing anything fancy at all. All I've added is Prabhu> a function that updates Tkinter when the timer is called Prabhu> and removed gtk's own mainloop with a hijacked version. The test is if arg1.endswith('-gthread'): so either -gthread or --gthread works. Hmm. I'm not experiencing any problems with iptyhon 0.6.4 or cvs ipython, but with the patched shell I'm getting very frequent freezes I spent a lot of time commenting out various parts of your patch to figure out what is causing the problems, and it appears to be def update_tk(tk): """Updates the Tkinter event loop. This is typically called from the respective WX or GTK mainloops. """ if tk: tk.update() If I simply return on this function rather than doing the update (or equivalently if I just set self.tk=None) I don't see the freeze. Some version info below. Note I am not otherwise using tk in my environment - could this be a source of the difference that we are seeing? Prabhu> [...] JH> FYI, I had to hack hijack_wx because on my wx, there is no JH> wx._core_. It appears to be wx._core. I don't know if a JH> similar hack needs to be applied to the 2.4 section. Prabhu> wxPython-2.4 is I think is more reliable and should work Prabhu> out fine. You'll need my updated patch that I just posted Prabhu> though (it just changes one line). I already applied it, thanks. Prabhu> To improve the hijacking can you just check these out and Prabhu> let me know? Does wx._core have a _core_ attribute? If Prabhu> so, then you really need to access wx._core._core_ and Prabhu> hijack that, this works for me also, so if this works for Prabhu> you, I'll make that the default and we won't need any of Prabhu> the hasattr checks. No, I don't have a wx._core._core_ (or anything like it) 3 >>> print [name for name in dir(wx._core) if name.startswith('_c')] [] 4 >>> print [name for name in dir(wx) if name.startswith('_c')] ['_controls', '_core'] Version info peds-pc311:~> uname -a Linux peds-pc311.bsd.uchicago.edu 2.4.21-15.0.2.ELsmp #1 SMP Wed Jun 16 22:52:07 EDT 2004 i686 i686 i386 GNU/Linux peds-pc311:~> python Python 2.3.3 (#2, Apr 13 2004, 17:41:29) [GCC 3.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import Tkinter >>> Tkinter.TkVersion 8.4000000000000004 From prabhu_r at users.sf.net Mon Nov 15 13:17:02 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Mon, 15 Nov 2004 23:47:02 +0530 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> Message-ID: <16792.62110.480414.977198@monster.linux.in> >>>>> "JH" == John Hunter writes: >>>>> "Prabhu" == Prabhu Ramachandran writes: JH> Hmm. I'm not experiencing any problems with iptyhon 0.6.4 or JH> cvs ipython, but with the patched shell I'm getting very JH> frequent freezes [...] Sorry about the problems. This is indeed quite wierd. I did not realize that the Tkinter updating could cause so many problems. JH> def update_tk(tk): [...] JH> Note I am not otherwise using tk in my environment - could JH> this be a source of the difference that we are seeing? Yep, that is the only possible problem we might be having, since apart from the hijacking, and the tk support there is nothing else my patch really does. [...] JH> No, I don't have a wx._core._core_ (or anything like it) JH> 3 >>> print [name for name in dir(wx._core) if JH> name.startswith('_c')] [] JH> 4 >>> print [name for name in dir(wx) if JH> name.startswith('_c')] ['_controls', '_core'] OK, so I assume that you do have a wx._core.PyApp_MainLoop attribute, right? JH> Version info JH> peds-pc311:~> uname -a JH> Linux peds-pc311.bsd.uchicago.edu 2.4.21-15.0.2.ELsmp #1 SMP JH> Wed Jun 16 22:52:07 EDT 2004 i686 i686 i386 GNU/Linux JH> peds-pc311:~> python JH> Python 2.3.3 (#2, Apr 13 2004, 17:41:29) [GCC 3.3.3] on linux2 JH> Type "help", "copyright", "credits" or "license" for more JH> information. >>>> import Tkinter Tkinter.TkVersion JH> 8.4000000000000004 I have the same exact Tkinter version but Python-2.3.4. $ uname -a Linux monster 2.4.20 #1 Mon Jan 6 19:56:44 IST 2003 i686 GNU/Linux $ python Python 2.3.4 (#2, Sep 24 2004, 08:39:09) [GCC 3.3.4 (Debian 1:3.3.4-12)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import Tkinter >>> Tkinter.TkVersion 8.4000000000000004 >>> Tkinter.TclVersion 8.4000000000000004 Anyway, it looks like Tkinter support does not work for everyone. This is wierd. Can you just check a few more things, please? $ ldd /usr/lib/python2.3/lib-dynload/_tkinter.so libBLT.2.4.so.8.4 => /usr/lib/libBLT.2.4.so.8.4 (0x4001f000) libtk8.4.so.0 => /usr/lib/libtk8.4.so.0 (0x4010a000) libtcl8.4.so.0 => /usr/lib/libtcl8.4.so.0 (0x401e3000) libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x40292000) libpthread.so.0 => /lib/libpthread.so.0 (0x40359000) libc.so.6 => /lib/libc.so.6 (0x403aa000) libm.so.6 => /lib/libm.so.6 (0x404dd000) libnsl.so.1 => /lib/libnsl.so.1 (0x40500000) libdl.so.2 => /lib/libdl.so.2 (0x40515000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) As you can see, mine links to libpthread. Similarly my tcl library also appears to be compiled with --enable-threads and links to libpthread. Is your version of Tcl compiled with thread support? $ ldd /usr/lib/libtcl8.4.so.0 libdl.so.2 => /lib/libdl.so.2 (0x400c3000) libpthread.so.0 => /lib/libpthread.so.0 (0x400c6000) libm.so.6 => /lib/libm.so.6 (0x40118000) libc.so.6 => /lib/libc.so.6 (0x4013a000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) Maybe Tkinter support should be made an option. Thanks for all the testing! cheers, prabhu From jdhunter at ace.bsd.uchicago.edu Mon Nov 15 13:20:29 2004 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Mon, 15 Nov 2004 12:20:29 -0600 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16792.62110.480414.977198@monster.linux.in> (Prabhu Ramachandran's message of "Mon, 15 Nov 2004 23:47:02 +0530") References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> Message-ID: >>>>> "Prabhu" == Prabhu Ramachandran writes: Prabhu> OK, so I assume that you do have a wx._core.PyApp_MainLoop Prabhu> attribute, right? Yes Prabhu> Anyway, it looks like Tkinter support does not work for Prabhu> everyone. This is wierd. Can you just check a few more Prabhu> things, please? Prabhu> $ ldd /usr/lib/python2.3/lib-dynload/_tkinter.so I don't have _tkinter.so. What does that mean? Perhaps my tk is compiled statically? peds-pc311:~> find /usr/local/lib/python2.3/ -iname "*tkin*" /usr/local/lib/python2.3/lib-tk/Tkinter.py /usr/local/lib/python2.3/lib-tk/Tkinter.pyc /usr/local/lib/python2.3/lib-tk/Tkinter.pyo Prabhu> As you can see, mine links to libpthread. Similarly my Prabhu> tcl library also appears to be compiled with Prabhu> --enable-threads and links to libpthread. Is your version Prabhu> of Tcl compiled with thread support? Prabhu> $ ldd /usr/lib/libtcl8.4.so.0 libdl.so.2 => Prabhu> /lib/libdl.so.2 (0x400c3000) libpthread.so.0 => Prabhu> /lib/libpthread.so.0 (0x400c6000) libm.so.6 => Prabhu> /lib/libm.so.6 (0x40118000) libc.so.6 => /lib/libc.so.6 Prabhu> (0x4013a000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 Prabhu> (0x80000000) This may be my problem - I don't seem to have pthread compiled in peds-pc311:~> ldd /usr/lib/libtk8.4.so libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0xb7452000) libdl.so.2 => /lib/libdl.so.2 (0xb7439000) libm.so.6 => /lib/tls/libm.so.6 (0xb7417000) libc.so.6 => /lib/tls/libc.so.6 (0xb72de000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) peds-pc311:~> ldd /usr/lib/libtcl8.4.so libdl.so.2 => /lib/libdl.so.2 (0xb7543000) libm.so.6 => /lib/tls/libm.so.6 (0xb7521000) libc.so.6 => /lib/tls/libc.so.6 (0xb73e9000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000) Prabhu> Maybe Tkinter support should be made an option. Prabhu> Thanks for all the testing! Prabhu> cheers, prabhu From Fernando.Perez at colorado.edu Mon Nov 15 13:40:56 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Mon, 15 Nov 2004 11:40:56 -0700 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16792.62110.480414.977198@monster.linux.in> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> Message-ID: <4198F838.8010502@colorado.edu> Hi guys, Prabhu Ramachandran wrote: >>>>>>"JH" == John Hunter writes: > > >>>>>>"Prabhu" == Prabhu Ramachandran writes: > > JH> Hmm. I'm not experiencing any problems with iptyhon 0.6.4 or > JH> cvs ipython, but with the patched shell I'm getting very > JH> frequent freezes > [...] > > Sorry about the problems. This is indeed quite wierd. I did not > realize that the Tkinter updating could cause so many problems. Sorry that I won't be able to help with real testing on this for a day or two, I'm a bit swamped. But I'm really excited about this working well in the end, so at least you have my cheers :) It would really be great to have tkinter functionality with any matplotlib backend, since for example mayavi is a key daily tool for many people (myself, for example). This would force me to use matplotlib with the Tk backend, which is not the nicest. But the freezes John is seeing are bad, and will cause end users no end of grief. I just saw John's message about not having tkinter built with threads, perhaps we can somehow detect this and make the code robust enough to at least not activate the feature if threads are not in? It's better to print a warning like "your Tk library has no thread support, so it can't coexist with WX/GTK" than to have strange lockups. Anyway, many thanks for this great work. I'll try to test it tomorrow further. At least as a data point: On my Fedora1 laptop: [IPython]> ldd /usr/lib/python2.2/lib-dynload/_tkinter.so libtix8.1.8.3.so => /usr/lib/libtix8.1.8.3.so (0x00be5000) libtk8.3.so => /usr/lib/libtk8.3.so (0x0021f000) libtcl8.3.so => /usr/lib/libtcl8.3.so (0x00b32000) libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x00111000) libc.so.6 => /lib/tls/libc.so.6 (0x00446000) libitk3.2.so => /usr/lib/libitk3.2.so (0x0094b000) libitcl3.2.so => /usr/lib/libitcl3.2.so (0x00fdb000) libdl.so.2 => /lib/libdl.so.2 (0x00812000) libm.so.6 => /lib/tls/libm.so.6 (0x00a7d000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00ce4000) On my Fedora2 desktop: planck[~]> ldd /usr/lib/python2.3/lib-dynload/_tkinter.so libtix8.1.8.4.so => /usr/lib/libtix8.1.8.4.so (0x55025000) libtk8.4.so => /usr/lib/libtk8.4.so (0x5505f000) libtcl8.4.so => /usr/lib/libtcl8.4.so (0x5512c000) libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x551cd000) libpthread.so.0 => /lib/tls/libpthread.so.0 (0x55295000) libc.so.6 => /lib/tls/libc.so.6 (0x552a7000) libdl.so.2 => /lib/libdl.so.2 (0x553c3000) libm.so.6 => /lib/tls/libm.so.6 (0x553c8000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x001bd000) These are both using the stock python. It appears that Fedora2 ships with pthreads, but NOT Fedora1. So I imagine I'd have John's problems on my laptop, but not on my desktop. Cheers, f From prabhu_r at users.sf.net Mon Nov 15 14:51:14 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Tue, 16 Nov 2004 01:21:14 +0530 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <4198F838.8010502@colorado.edu> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> <4198F838.8010502@colorado.edu> Message-ID: <16793.2226.560607.215628@monster.linux.in> >>>>> "FP" == Fernando Perez writes: FP> On my Fedora1 laptop: [...] FP> On my Fedora2 desktop: [...] FP> These are both using the stock python. It appears that FP> Fedora2 ships with pthreads, but NOT Fedora1. So I imagine FP> I'd have John's problems on my laptop, but not on my desktop. John also has the problem that the _tkinter.so does not exist (not that it might matter). So a confirmation of this with a quick test will definitely help. Please do this whenever you can. I don't think there is going to be an easy way to detect the usuability of Tkinter with wthread/gthread. I think the best solution is to add another option so that Tk support can be turned on or off on the command line. An easy way to do this would be to add a global variable that is turned on via sys.argv[2]. Something like so:: ipython -wthread [-tk] ipython -gthread [-tk] So if the user wants to try tkinter support they try with the option. If they really want it to work they either move to Debian or rebuild Python/Tcl appropriately. ;-) I think that should do fine enough. Those who never use Tkinter apps don't have to bother at all with the -tk option. Thoughts? Thanks. prabhu From jdhunter at ace.bsd.uchicago.edu Mon Nov 15 14:47:32 2004 From: jdhunter at ace.bsd.uchicago.edu (John Hunter) Date: Mon, 15 Nov 2004 13:47:32 -0600 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16793.2226.560607.215628@monster.linux.in> (Prabhu Ramachandran's message of "Tue, 16 Nov 2004 01:21:14 +0530") References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> <4198F838.8010502@colorado.edu> <16793.2226.560607.215628@monster.linux.in> Message-ID: >>>>> "Prabhu" == Prabhu Ramachandran writes: Prabhu> ipython -wthread [-tk] ipython -gthread [-tk] Prabhu> So if the user wants to try tkinter support they try with Prabhu> the option. If they really want it to work they either Prabhu> move to Debian or rebuild Python/Tcl appropriately. ;-) I Prabhu> think that should do fine enough. Those who never use Prabhu> Tkinter apps don't have to bother at all with the -tk Prabhu> option. Following explicit is better than implicit, I think it should be a flag. ipython should warn if -gthread and -wthread are both given, but otherwise accept combinations of -[gw]thread and -tk. And the docs of course should mention that tk requires threading built in and then it should work great. JDH From prabhu_r at users.sf.net Mon Nov 15 15:10:13 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Tue, 16 Nov 2004 01:40:13 +0530 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> Message-ID: <16793.3365.879383.969444@monster.linux.in> >>>>> "JH" == John Hunter writes: >>>>> "Prabhu" == Prabhu Ramachandran writes: Prabhu> OK, so I assume that you do have a wx._core.PyApp_MainLoop Prabhu> attribute, right? JH> Yes Thanks. JH> I don't have _tkinter.so. What does that mean? Perhaps my tk JH> is compiled statically? ldd on the Python executable and try. [...] JH> This may be my problem - I don't seem to have pthread compiled JH> in [...] JH> peds-pc311:~> ldd /usr/lib/libtcl8.4.so JH> libdl.so.2 => /lib/libdl.so.2 (0xb7543000) libm.so.6 JH> => /lib/tls/libm.so.6 (0xb7521000) libc.so.6 => JH> /lib/tls/libc.so.6 (0xb73e9000) /lib/ld-linux.so.2 => JH> /lib/ld-linux.so.2 (0x80000000) Yes, that is a possibility. Actually, since the error is random it might be a little subtler. update_tk() is run outside of runcode. So this might be breaking thread safety somehow. I've attached a modified Shell.py that tries to do the right thing by calling update_tk from inside runcode. Please let me know if it gets any better and if there are no freezes anymore. This *might* just do the trick. Thanks! cheers, prabhu -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: Shell.py URL: From Fernando.Perez at colorado.edu Mon Nov 15 15:19:16 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Mon, 15 Nov 2004 13:19:16 -0700 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16793.3365.879383.969444@monster.linux.in> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> <16793.3365.879383.969444@monster.linux.in> Message-ID: <41990F44.4030600@colorado.edu> Prabhu Ramachandran schrieb: > Yes, that is a possibility. Actually, since the error is random it > might be a little subtler. update_tk() is run outside of runcode. So > this might be breaking thread safety somehow. I've attached a > modified Shell.py that tries to do the right thing by calling > update_tk from inside runcode. Please let me know if it gets any > better and if there are no freezes anymore. This *might* just do the > trick. Quick report with this Shell.py you just sent. On a Fedora3 box, I get with -wthread: 1. no immediate lockup 2. I can start mayavi from the ipython shell, and load a visualization: In [1]: import mayavi In [2]: mv = mayavi.mayavi() In [3]: mv.load_visualization('heart.mv') 3. BUT, the mayavi renderwindow responds very sluggishly. Rotating the view is very jerky, which it normally isn't if I don't use the -wthread option (it's actually smooth as silk). 4. I kept rotating the view, and all of a sudden python coredumped. Mayavi and ipython both went up in smoke. Not good. This is definitely not user-ready, sorry to say so :( Cheers, f From Fernando.Perez at colorado.edu Mon Nov 15 19:27:24 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Mon, 15 Nov 2004 17:27:24 -0700 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16792.34276.631159.477706@monster.linux.in> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> Message-ID: <4199496C.1000900@colorado.edu> Prabhu Ramachandran schrieb: > BTW, Fernando, did you get to post the modified python-mode.el to the > python-mode maintainers? I sent it to the address you suggested, but I never heard back from anyone there. I'm pasting below what I sent (without the attachment), perhaps you can bang on their door once more... Best, f. ----------------------------------------------------------- Dear Python-mode maintainers, the attached code is a patched version of python-mode 4.62, which incorporates some changes to allow python-mode to support ipython[1] instead of the default python interpreter in (X)emacs. The work was done by Alex Schmolck and Prabhu Ramachandran, and over time ipython has distributed this specially patched version (since python-mode 4.22) to support ipytho's users. It would be great if these changes could be folded into the python-mode mainline code. As far as I understand, the changes are fairly unobtrusive for normal python users, and they would benefit greatly ipython's user base. Best regards, Fernando Perez [1] http://ipython.scipy.org Below follows the original message from Prabhu regarding this work. -------- Original Message -------- Subject: Re: [IPython-user] ipython + emacs py-execute-region problem. Date: Fri, 15 Oct 2004 15:23:25 +0530 From: Prabhu Ramachandran To: Fernando Perez CC: Bo Peng , ipython-user at scipy.net References: <6ea7b543041014093632091eb at mail.gmail.com> <416F6CB9.9040005 at colorado.edu> <16751.32181.153111.604247 at monster.linux.in> <416F7EEC.9010709 at colorado.edu> >>>>>>>>>> "FP" == Fernando Perez writes: >> Well, I think Bo means that this does not seem to happen if he >> uses Alex's python-mode but does not use ipython. So it >> appears that something funky happens when ipython is used. I >> am not elisp guru either to be able to help. When will Alex be >> back? FP> Oh, I misunderstood things then. My point about the lag in FP> the ipython-compatible version of python-mode being an FP> unpleaseant situation remains, though. FP> I think Alex said a month, and he only left a few days ago. Nevermind, I have fixed the problem. I've updated python-mode.el to the latest version (revision 4.62) available from CVS here: http://cvs.sourceforge.net/viewcvs.py/python-mode/python-mode/ I've retained Alex's fixes for IPython support in this version. I have also added a defcustom for py-shell-switch-buffers-on-execute which defaults to true and will therefore switch buffers to the py-shell when you execute a region. If you turn it off it disables this behavior. So, switching the buffer was a feature and not a bug, it appears. So, to get Bo's desired behavior you simply need to do this in your .xemacs/init.el: (require 'python-mode) (setq py-shell-switch-buffers-on-execute nil) (require 'ipython) Please find attached a gzipped file containing the new-and-improved python-mode.el. I think you could consider sending this in to python-mode at python.org. I'm not doing it since I have already spent a little too much time on this today. cheers, prabhu From prabhu_r at users.sf.net Mon Nov 15 22:31:22 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Tue, 16 Nov 2004 09:01:22 +0530 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <41990F44.4030600@colorado.edu> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> <16793.3365.879383.969444@monster.linux.in> <41990F44.4030600@colorado.edu> Message-ID: <16793.29834.831423.950877@monster.linux.in> >>>>> "FP" == Fernando Perez writes: [...] FP> Quick report with this Shell.py you just sent. On a Fedora3 FP> box, I get with -wthread: [...] FP> 4. I kept rotating the view, and all of a sudden python FP> coredumped. Mayavi and FP> ipython both went up in smoke. Not good. FP> This is definitely not user-ready, sorry to say so :( Yeah, I thought as much. I realized later that the new Shell.py was a late night shot-in-the-dark. On my debian machine, everything works nicely (no jerking, no crashing). So, this is doubly frustrating. Why doesn't everyone run debian? ;-) BTW, does this version of FC ship with threaded versions of libtcl/libtk/_tkinter? If it does, I am at a loss. Anyway, I think the only way out of this mess would be to add a -tk flag. I've just done that. So you can do -{g,w}thread without -tk and you should see no problems. If you have decent Tcl/Tk/tkinter build or a good OS ;-) then you could risk trying '-tk'. I've made a patch for this and attached it below. The code isn't the greatest but works for me. So to enable Tk do `ipython -{g,w}thread -tk`. cheers, prabhu -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ipython.patch URL: From prabhu_r at users.sf.net Mon Nov 15 22:47:36 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Tue, 16 Nov 2004 09:17:36 +0530 Subject: [IPython-dev] python-mode.el (was Generic gui_thread + IPython: solution already exists!) In-Reply-To: <4199496C.1000900@colorado.edu> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <4199496C.1000900@colorado.edu> Message-ID: <16793.30808.955514.682405@monster.linux.in> >>>>> "FP" == Fernando Perez writes: FP> Prabhu Ramachandran schrieb: >> BTW, Fernando, did you get to post the modified python-mode.el >> to the python-mode maintainers? FP> I sent it to the address you suggested, but I never heard back Thanks! FP> from anyone there. I'm pasting below what I sent (without the FP> attachment), perhaps you can bang on their door once more... Well, your ancient 2003 patch is in the patch queue on the SF tracker. http://sourceforge.net/tracker/index.php?func=detail&aid=783254&group_id=86916&atid=581351 I think the best approach would be to mail Skip Montanaro (or ) directly and ask what one has to do to get the darned thing applied. Its a pain to keep updating our version of python-mode as they make changes to it. I'm going to write to him now and will CC you. cheers, prabhu From Fernando.Perez at colorado.edu Mon Nov 15 23:03:52 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Mon, 15 Nov 2004 21:03:52 -0700 Subject: [IPython-dev] Re: python-mode.el (was Generic gui_thread + IPython: solution already exists!) In-Reply-To: <16793.30808.955514.682405@monster.linux.in> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <4199496C.1000900@colorado.edu> <16793.30808.955514.682405@monster.linux.in> Message-ID: <41997C28.5020107@colorado.edu> Prabhu Ramachandran wrote: > Well, your ancient 2003 patch is in the patch queue on the SF tracker. > > http://sourceforge.net/tracker/index.php?func=detail&aid=783254&group_id=86916&atid=581351 Huh, that patch seems to have Skip's name on it, not mine. And I don't quite recall having ever submitted a patch to python-mode (though I might have forgotten). Are you sure it's mine? > I think the best approach would be to mail Skip Montanaro > (or ) directly and ask what one > has to do to get the darned thing applied. Its a pain to keep > updating our version of python-mode as they make changes to it. > > I'm going to write to him now and will CC you. At any rate, this is good. I hope they pick that ball up, it's a silly duplication of effort on our part, and we have more than enough fish to fry around here. Many thanks, f From Fernando.Perez at colorado.edu Mon Nov 15 23:13:33 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Mon, 15 Nov 2004 21:13:33 -0700 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16793.29834.831423.950877@monster.linux.in> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> <16793.3365.879383.969444@monster.linux.in> <41990F44.4030600@colorado.edu> <16793.29834.831423.950877@monster.linux.in> Message-ID: <41997E6D.60108@colorado.edu> Prabhu Ramachandran wrote: >>>>>>"FP" == Fernando Perez writes: > > > [...] > FP> Quick report with this Shell.py you just sent. On a Fedora3 > FP> box, I get with -wthread: > > [...] > FP> 4. I kept rotating the view, and all of a sudden python > FP> coredumped. Mayavi and > FP> ipython both went up in smoke. Not good. > > FP> This is definitely not user-ready, sorry to say so :( > > Yeah, I thought as much. I realized later that the new Shell.py was a > late night shot-in-the-dark. > > On my debian machine, everything works nicely (no jerking, no > crashing). So, this is doubly frustrating. Why doesn't everyone run > debian? ;-) > > BTW, does this version of FC ship with threaded versions of > libtcl/libtk/_tkinter? If it does, I am at a loss. Well, here's some more info: planck[~]> ldd /usr/lib/libtk8.4.so libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x0031f000) libdl.so.2 => /lib/libdl.so.2 (0x00319000) libm.so.6 => /lib/tls/libm.so.6 (0x002f4000) libc.so.6 => /lib/tls/libc.so.6 (0x001d6000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x001bd000) planck[~]> ldd /usr/lib/libtcl8.4.so libdl.so.2 => /lib/libdl.so.2 (0x00319000) libm.so.6 => /lib/tls/libm.so.6 (0x002f4000) libc.so.6 => /lib/tls/libc.so.6 (0x001d6000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x001bd000) planck[~]> ldd /usr/lib/python2.3/lib-dynload/_tkinter.so libtix8.1.8.4.so => /usr/lib/libtix8.1.8.4.so (0x55025000) libtk8.4.so => /usr/lib/libtk8.4.so (0x5505f000) libtcl8.4.so => /usr/lib/libtcl8.4.so (0x5512c000) libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x551cd000) libpthread.so.0 => /lib/tls/libpthread.so.0 (0x55295000) libc.so.6 => /lib/tls/libc.so.6 (0x552a7000) libdl.so.2 => /lib/libdl.so.2 (0x553c3000) libm.so.6 => /lib/tls/libm.so.6 (0x553c8000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x001bd000) So it seems that _tkinter has thread support, but NOT tcl/tk. Maybe that explains the difference with your box. Unfortunately, I can't really offer to do a full separate build of tcl/tk/tkinter/python to get a fully threaded setup, that would take _waaaay_ more time than I can spend on this. But it does piss me off to know that it's a stupid distribution issue! Your debian success proves that it _can_ work! Do you know of any reason why Fedora would do the tcl/tk builds without thread support, other than moronism? > Anyway, I think the only way out of this mess would be to add a -tk > flag. I've just done that. So you can do -{g,w}thread without -tk > and you should see no problems. If you have decent Tcl/Tk/tkinter > build or a good OS ;-) then you could risk trying '-tk'. I've made a > patch for this and attached it below. The code isn't the greatest but > works for me. So to enable Tk do `ipython -{g,w}thread -tk`. Yes, the -tk separate flag would certainly be a necessity. I can always fold this into offering -gthread AND -gthreadtk (similar for -wthread), so people only need to pass a single option to activate all thread/gui stuff. I can unpack them inside myself. But this will have to be strongly documented, since it's so easy to get a broken, segfaulting setup. I'm seriously starting to envy debian... Thanks again! Do you think you'll tweak this further, or should I consider this last patch final? I'm in no hurry. Best, f From prabhu_r at users.sf.net Tue Nov 16 00:34:23 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Tue, 16 Nov 2004 11:04:23 +0530 Subject: [IPython-dev] Re: python-mode.el (was Generic gui_thread + IPython: solution already exists!) In-Reply-To: <41997C28.5020107@colorado.edu> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <4199496C.1000900@colorado.edu> <16793.30808.955514.682405@monster.linux.in> <41997C28.5020107@colorado.edu> Message-ID: <16793.37215.882154.584956@monster.linux.in> >>>>> "FP" == Fernando Perez writes: FP> Prabhu Ramachandran wrote: >> Well, your ancient 2003 patch is in the patch queue on the SF >> tracker. >> >> http://sourceforge.net/tracker/index.php?func=detail&aid=783254&group_id=86916&atid=581351 FP> Huh, that patch seems to have Skip's name on it, not mine. FP> And I don't quite recall having ever submitted a patch to FP> python-mode (though I might have forgotten). Are you sure FP> it's mine? I think it was a patch submitted to Python and moved to the python-mode project by Skip in 2003. cheers, prabhu From prabhu_r at users.sf.net Tue Nov 16 01:57:21 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Tue, 16 Nov 2004 12:27:21 +0530 Subject: [IPython-dev] Re: [SciPy-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <41997E6D.60108@colorado.edu> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> <16793.3365.879383.969444@monster.linux.in> <41990F44.4030600@colorado.edu> <16793.29834.831423.950877@monster.linux.in> <41997E6D.60108@colorado.edu> Message-ID: <16793.42193.462547.226510@monster.linux.in> >>>>> "FP" == Fernando Perez writes: >> BTW, does this version of FC ship with threaded versions of >> libtcl/libtk/_tkinter? If it does, I am at a loss. FP> Well, here's some more info: FP> planck[~]> ldd /usr/lib/libtk8.4.so [...] FP> So it seems that _tkinter has thread support, but NOT tcl/tk. FP> Maybe that explains the difference with your box. FP> Unfortunately, I can't really offer to do a full separate FP> build of tcl/tk/tkinter/python to get a fully threaded setup, FP> that would take _waaaay_ more time than I can spend on this. Hmm, actually, it looks like there are even subtler issues! Darn! >From what I understand, only when thread.start() is invoked does the threading actually begin. So in all my earlier code Tkinter is imported (along with a Tcl interpreter underneath) inside the main thread and NOT the gui_thread! This is what caused all the headache for you. I tried a simple fix for this but it looks like if Tcl is itself threaded (as in my case) it must be in the main thread! Thus, I'd need to make two different versions, one that works on threaded tk's and one on non-threaded tk's. Pain! I have to run now. Will try to send a patch tonight so you can test it out for the non-threaded setups. FP> But it does piss me off to know that it's a stupid FP> distribution issue! Your debian success proves that it _can_ FP> work! Do you know of any reason why Fedora would do the FP> tcl/tk builds without thread support, other than moronism? No clue. >> Anyway, I think the only way out of this mess would be to add a >> -tk flag. I've just done that. So you can do -{g,w}thread >> without -tk and you should see no problems. If you have decent >> Tcl/Tk/tkinter build or a good OS ;-) then you could risk >> trying '-tk'. I've made a patch for this and attached it >> below. The code isn't the greatest but works for me. So to >> enable Tk do `ipython -{g,w}thread -tk`. FP> Yes, the -tk separate flag would certainly be a necessity. I FP> can always fold this into offering -gthread AND -gthreadtk FP> (similar for -wthread), so people only need to pass a single FP> option to activate all thread/gui stuff. I can unpack them FP> inside myself. But this will have to be strongly documented, FP> since it's so easy to get a broken, segfaulting setup. Yes. FP> I'm seriously starting to envy debian... :) FP> Thanks again! Do you think you'll tweak this further, or FP> should I consider this last patch final? I'm in no hurry. No, not final. We need to do more stunts here. I'll mail the list a patch tonight that I'd appreciate if someone with a non-threaded Tcl can test with. Then we can change things suitably. cheers, prabhu From Fernando.Perez at colorado.edu Tue Nov 16 01:57:57 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Mon, 15 Nov 2004 23:57:57 -0700 Subject: [IPython-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <16793.42193.462547.226510@monster.linux.in> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> <16793.3365.879383.969444@monster.linux.in> <41990F44.4030600@colorado.edu> <16793.29834.831423.950877@monster.linux.in> <41997E6D.60108@colorado.edu> <16793.42193.462547.226510@monster.linux.in> Message-ID: <4199A4F5.8090601@colorado.edu> Prabhu Ramachandran wrote: > No, not final. We need to do more stunts here. I'll mail the list a > patch tonight that I'd appreciate if someone with a non-threaded Tcl > can test with. Then we can change things suitably. Send it this way, I can test it tomorrow briefly in my office (the fedora2 destkop with threaded _tkinter but non-threaded tcl/tk). Cheers, f From Fernando.Perez at colorado.edu Tue Nov 16 02:01:32 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Tue, 16 Nov 2004 00:01:32 -0700 Subject: [IPython-dev] Re: python-mode.el (was Generic gui_thread + IPython: solution already exists!) In-Reply-To: <16793.37215.882154.584956@monster.linux.in> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <4199496C.1000900@colorado.edu> <16793.30808.955514.682405@monster.linux.in> <41997C28.5020107@colorado.edu> <16793.37215.882154.584956@monster.linux.in> Message-ID: <4199A5CC.3020803@colorado.edu> Prabhu Ramachandran wrote: >>>>>>"FP" == Fernando Perez writes: > > > FP> Prabhu Ramachandran wrote: > >> Well, your ancient 2003 patch is in the patch queue on the SF > >> tracker. > >> > >> http://sourceforge.net/tracker/index.php?func=detail&aid=783254&group_id=86916&atid=581351 > > FP> Huh, that patch seems to have Skip's name on it, not mine. > FP> And I don't quite recall having ever submitted a patch to > FP> python-mode (though I might have forgotten). Are you sure > FP> it's mine? > > I think it was a patch submitted to Python and moved to the > python-mode project by Skip in 2003. Ah, ok. Quite possible that I just forgot. Anyway, many thanks for following this through! I think I'll now include ipython.el with the distro, since it's such a small file. Do you have any advice on where I should target it for installation with distutils? I may just punt and drop it in the examples directory, leaving it to the users to put it in their respective emacs location, but if there's a good canonical solution, I wouldn't mind doing the right thing from the start. Best, f From a.schmolck at gmx.net Tue Nov 16 06:48:39 2004 From: a.schmolck at gmx.net (Alexander Schmolck) Date: Tue, 16 Nov 2004 11:48:39 +0000 Subject: [IPython-dev] Re: python-mode.el (was Generic gui_thread + IPython: solution already exists!) In-Reply-To: <41997C28.5020107@colorado.edu> (Fernando Perez's message of "Mon, 15 Nov 2004 21:03:52 -0700") References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <4199496C.1000900@colorado.edu> <16793.30808.955514.682405@monster.linux.in> <41997C28.5020107@colorado.edu> Message-ID: Fernando Perez writes: > Prabhu Ramachandran wrote: > >> Well, your ancient 2003 patch is in the patch queue on the SF tracker. >> http://sourceforge.net/tracker/index.php?func=detail&aid=783254&group_id=86916&atid=581351 > > Huh, that patch seems to have Skip's name on it, not mine. It's mine and the reason it has Skip's name on it is that it presumably predates 2003 and the project page and was one of the old patches that were migrated from the python sf project to the new python-mode sf project. > And I don't quite recall having ever submitted a patch to python-mode > (though I might have forgotten). Are you sure it's mine? As I said it isn't. That patch referred to above will almost certainly not work for newest CVS python -- at least it didn't many months ago. Has someone since then created a new version of the patch? > >> I think the best approach would be to mail Skip Montanaro >> (or ) directly and ask what one >> has to do to get the darned thing applied. Its a pain to keep >> updating our version of python-mode as they make changes to it. >> I'm going to write to him now and will CC you. While we're at it I've also fixed what I take to be 2 other stupidities in python mode: - eliminated an almost certainly spurious (interactive-p); IIRC the reason I did that is that it it broke C-u C-c! (i.e. starting py-shell so that it prompts for arguments which is useful to e.g. to start ipython with a certain profile (by passing "-p some_profile"), without hardcoding it somewhere) - calling M-x py-shell (viz C-c!) in *Python* now doesn't switch'es-to-buffer-other-window because if one is already in the *Python* buffer than that's almost certainly *not* what one want. I've just made a patch against the newest CVS python and I'll attach it to the message (because the desire to call ipython from emacs with a certain profile has been expressed here a few times and doing it with C-u py-shell). Of course this really belongs to the python-mode patch tracker, but before I spend time submitting anything there I really want to see some feedback on the ipython-compatibility patch originally submitted by me years ago and now apparently/maybe (?) updated by someone else. > At any rate, this is good. I hope they pick that ball up, it's a silly > duplication of effort on our part, and we have more than enough fish to fry > around here. Indeed. Provided there is a version of the patch that works against CVS python-mode.el than I think ipython/ipython-mode.el by now have a large enough user base that at least giving some reason for *not* accepting the patch wouldn't seem to much to ask for. If there isn't I'll try to help creating one provided someone provides me with an assurance that it will be accepted (if it fullfils some reasonable criteria). 'as -------------- next part -------------- A non-text attachment was scrubbed... Name: python-mode.el.PATCH Type: text/x-patch Size: 1043 bytes Desc: not available URL: From prabhu_r at users.sf.net Tue Nov 16 10:02:24 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Tue, 16 Nov 2004 20:32:24 +0530 Subject: [IPython-dev] Re: python-mode.el (was Generic gui_thread + IPython: solution already exists!) In-Reply-To: References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <4199496C.1000900@colorado.edu> <16793.30808.955514.682405@monster.linux.in> <41997C28.5020107@colorado.edu> Message-ID: <16794.5760.739476.821885@monster.linux.in> >>>>> "AS" == Alexander Schmolck writes: >> Huh, that patch seems to have Skip's name on it, not mine. AS> It's mine and the reason it has Skip's name on it is that it AS> presumably predates 2003 and the project page and was one of AS> the old patches that were migrated from the python sf project AS> to the new python-mode sf project. Welcome back! We missed you. ;-) When you were away, I stumbled my way through python-mode.el and updated python-mode.el.gz to 4.62 from CVS. Look at the archives to see the messages. Just last night I asked Skip Montanaro for help since there was no reply to Fernando's mail containing the new python-mode.el. Skip responded really quickly with a patch that incorporated all the features that were there in the older version. I've attached the diff he sent me (against 4.67 in CVS). I replied early today morning (GMT+530) and told him that the patch looks good. So your code should be integrated in python-mode.el very soon. [...] AS> While we're at it I've also fixed what I take to be 2 other AS> stupidities in python mode: [...] Hmm, you might want to mention these to Skip if they belong in python-mode.el BTW, switch to buffer on C-c| (py-execute-region) is now optional via a defcustom variable. Anyway, please find attached the patch from Skip to me. This should incoporate your original changes to python-mode.el (and then some) and works with python-mode.el from CVS version 4.67. cheers, prabhu -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ipy.diff URL: From prabhu_r at users.sf.net Tue Nov 16 14:41:39 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Wed, 17 Nov 2004 01:11:39 +0530 Subject: [IPython-dev] Generic gui_thread + IPython: solution already exists! In-Reply-To: <4199A4F5.8090601@colorado.edu> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <16792.53672.53562.888095@monster.linux.in> <16792.56292.811530.847847@monster.linux.in> <16792.62110.480414.977198@monster.linux.in> <16793.3365.879383.969444@monster.linux.in> <41990F44.4030600@colorado.edu> <16793.29834.831423.950877@monster.linux.in> <41997E6D.60108@colorado.edu> <16793.42193.462547.226510@monster.linux.in> <4199A4F5.8090601@colorado.edu> Message-ID: <16794.22515.789332.986978@monster.linux.in> >>>>> "FP" == Fernando Perez writes: FP> Prabhu Ramachandran wrote: >> No, not final. We need to do more stunts here. I'll mail the >> list a patch tonight that I'd appreciate if someone with a >> non-threaded Tcl can test with. Then we can change things >> suitably. FP> Send it this way, I can test it tomorrow briefly in my office FP> (the fedora2 destkop with threaded _tkinter but non-threaded FP> tcl/tk). Well, I spent quite bit of time on this. I can't really see why this is a problem on non-threaded Tcl's. This is how the gtk/wx threading works: 1. gtk/wx mainloop is running in the main thread. 2. The timer is called from the main thread and this calls runcode to interpret the Python code. 3. The interpreter console runs in the secondary thread and interacts with the user on the console. Now, if I import Tkinter in the gtk/wx thread, then the Tcl interpreter is "running" in the main thread. `update_tk(...)` is called in the main thread by the timer. So, why is there a problem at all? With your current Shell.py, what happens if you do this:: ipython -gthread # or -wthread >>> import Tkinter >>> r = Tkinter.Tk() >>> r.update() >>> import mayavi >>> v = mayavi.mayavi() >>> v.load_visualization('some_mv_file.mv') >>> r.update() This is essentially all I am doing in the patch. The only difference is that r.update() is called automatically by the timer callback. Anyway, I think I'm done with this. It seems to work pretty well for me under Debian. So with the -tk option I think, this is reasonably good enough for now. cheers, prabhu From prabhu_r at users.sf.net Tue Nov 16 15:00:27 2004 From: prabhu_r at users.sf.net (Prabhu Ramachandran) Date: Wed, 17 Nov 2004 01:30:27 +0530 Subject: [IPython-dev] Re: python-mode.el (was Generic gui_thread + IPython: solution already exists!) In-Reply-To: <4199A5CC.3020803@colorado.edu> References: <16791.45358.186631.80855@monster.linux.in> <4197D40C.6060103@colorado.edu> <16792.5872.782287.158081@monster.linux.in> <41984089.1070005@colorado.edu> <16792.34276.631159.477706@monster.linux.in> <4199496C.1000900@colorado.edu> <16793.30808.955514.682405@monster.linux.in> <41997C28.5020107@colorado.edu> <16793.37215.882154.584956@monster.linux.in> <4199A5CC.3020803@colorado.edu> Message-ID: <16794.23643.160097.696899@monster.linux.in> >>>>> "FP" == Fernando Perez writes: [...] FP> small file. Do you have any advice on where I should target FP> it for installation with distutils? I may just punt and drop FP> it in the examples directory, leaving it to the users to put FP> it in their respective emacs location, but if there's a good FP> canonical solution, I wouldn't mind doing the right thing from FP> the start. Perhaps make a 'doc/emacs' directory and stick it there? prabhu From Fernando.Perez at colorado.edu Tue Nov 30 19:25:44 2004 From: Fernando.Perez at colorado.edu (Fernando Perez) Date: Tue, 30 Nov 2004 17:25:44 -0700 Subject: [IPython-dev] ANN: IPython 0.6.5 is out. Message-ID: <41AD0F88.6060709@colorado.edu> Hi all, I'm glad to announce that IPython 0.6.5 is finally out. You can download it from the usual place: http://ipython.scipy.org/dist/ I've provided RPMs for Python 2.2 and 2.3, plus source downloads (.tar.gz and .zip). Debian, Fink and BSD packages for this version should be coming soon, as the respective maintainers (many thanks to Jack Moffit, Andrea Riciputi and Dryice Liu) have the time to follow their packaging procedures. Many thanks to Enthought for their continued hosting support for IPython, and to all the users who contributed ideas, fixes and reports. I'd promised that 0.6.4 would be the last version before the cleanup, but Prabhu Ramachandran managed to resucitate the GUI threading support which I'd worked on recently, but disabled after thinking it could not work. It turns out we were very close, and Prabhu did fix the remaining problems. Since this is a fairly significant improvement, I decided to make a release for it. In the process I added a few minor other things. As always, the complete NEWS file can be found at http://ipython.scipy.org/NEWS, and the whole ChangeLog at http://ipython.scipy.org/ChangeLog. Highlihgts of this release: * Threading support for WXPython and pyGTK. It is now possible (with the -wthread and -gthread options) to control wx/gtk graphical interfaces from within an interactive ipython shell. Note that your wx/gtk libs need to be compiled with threading support for this to work. There is also experimental (but brittle) support for ALSO running Tkinter graphical interfaces alongside with wx or gtk ones. * New -d option to %run, for executing whole scripts with the interactive pdb debugger. This allows you to step, watch variables, set breakpoints, etc, without having to modify your scripts in any way. * Added filtering support for variable types to %who and %whos. You can now say 'whos function str' and whos will only list functions and strings, instead of all variables. Useful when working with crowded namespaces. (For some reason I forgot to document this in the ChangeLog). * Other minor fixes and cleanups, both to code and documentation. Enjoy, and as usual please report any problems. Regards, Fernando.