[Python-Dev] Event loops, PyOS_InputHook, and Tkinter - Summary attempt

[Fernando Perez]

Jim Jewett wrote:

> (6)  Mark Hammond suggests that it might be easier to
> replace the interactive portions of python based on the
> "code" module.  matplotlib suggests using ipython
> instead of standard python for similar reasons.
> 
> If that is really the simplest answer (and telling users
> which IDE to use is acceptable), then ... I think Michiel
> has a point.

I don't claim to understand all the low-level details of this discussion, by a
very long shot.  But as the author of ipython, at least I'll mention what
ipython does to help in this problem.  Whether that is a satisfactory solution
for everyone or not, I won't get into.

For starters, ipython is an extension of the code.InteractiveConsole class, even
though by now I've changed so much that I could probably just stop using any
inheritance at all.  But this is just to put ipython in the context of the
stdlib.  

When I started using matplotlib, I wanted to be able to run my code
interactively and get good plotting, as much as I used to have with Gnuplot
before (IPython ships with extended Gnuplot support beyond what the default
Gnuplot.py module provides).  With help from John Hunter (matplotlib - mpl for
short - author), we were able to add support for ipython to happily coexist
with matplotlib when either the GTK or the WX backends were used. mpl can plot
to Tk, GTK, WX, Qt and FLTK; Tk worked out of the box (because of the Tkinter
event loop integration in Python), and with our hacks we got GTK and WX to
work.  Earlier this year, with the help of a few very knowledgeable Qt
developers, we extended the same ideas to add support for Qt as well.  As part
of this effort, ipython can generically (meaning, outside of matplotlib)
support interactive non-blocking control of WX, GTK and Qt apps, you get that
by starting it with

ipython -wthread/-gthread/-qthread

The details of how this works are slightly different for each toolkit, but the
overall approach is the same for all.  We just register with each toolkit's
idle/timer system a callback to execute pending code which is waiting in what
is essentially a one-entry queue.  I have a private branch where I'm adding
similar support for OpenGL windows using the GLUT idle function, though it's ot
ready for release yet.  So far this has worked quite well.  
If anyone wants to see the details, the relevant code is here:

http://projects.scipy.org/ipython/ipython/file/ipython/trunk/IPython/Shell.py

It may not be perfect, and it may well be the wrong approach.  If so, I'll be
glad to learn how to do it better: I know very little about threading and I got
this to work more or less by stumbling in the dark.

In particular, one thing that definitely does NOT work is mixing TWO GUI
toolkits together.  There is a hack (the -tk option) to try to allow mixing of
ONE of Qt/WX/GTK with Tk, but it has only ever worked on Debian, and we don't
really understand why.  I think it's some obscure combination of how the
low-level threading support for many different libraries is compiled in Debian.

As far as using IDLE/Emacs/whatever (I use Xemacs personally for my own
editing), our approach has been to simply tell people that the _interactive
shell_ should be ipython always.  They can use anything they want to edit their
code with, but they should execute their scripts with ipython.  ipython has a
%run command which allows code execution with a ton of extra control, so the
work cycle with ipython is more or less:

1. open the editor you like to use with your foo.py code.  Hack on foo.py

2. whenever you wish to test your code, save foo.py

3. switch to the ipython window, and type 'run foo'.  Play with the results
interactively (the foo namespace updates the interactive one after completion).

4. rinse, repeat.

In the matplotlib/scipy mailing lists we've more or less settled on 'this is
what we support.  If you don't like it, go write your own'.  It may not be
perfect, but it works reasonably for us (I use this system 10 hours a day in
scientific production work, and so does John, so we do eat our own dog food).

Given that ipython is trival to install (it's pure python code with no extra
dependencies under *nix and very few under win32), and that it provides so much
additional functionality on top of the default interactive interpreter, we've
had no complaints so far.

OK, I hope this information is useful to some of you.  Feel free to contact me
if you have any questions (I will monitor the thread, but I follow py-dev on
gmane, so I do miss things sometimes).

Cheers,

f