[IPython-dev] PyMAD/IPython which list updated

[Fernando Perez]

Thanks for this update.  Below follow a few more comments, it would be great 
if you could keep the doc updated as we hash out the issues.  I'm thinking a 
wiki would be great for this, but for now we don't have one.  So while I code, 
you can be the FrédéWiki ;-)  I actually think most things will be cleared up 
in short order, so in reality there's no need to do much.

Frédéric Mantegazza wrote:

>    1.
> 
>       Catch custom /PyMADError/ exceptions (now done with rebinding
>       IPython.iplib.InteractiveShell.runcode() method), *with the possibility to
>       get the inital text code involved in the exception*. For the moment, in
>       the runcode() method, we only get the code object, from which it is
>       impossible to retreive the text. Here is the code used:

OK, now, IPython's runsource looks as follows:

         # Case 3
         # We store the code source and object so that threaded shells and
         # custom exception handlers can access all this info if needed.
         self.code_to_run_src = source
         self.code_to_run = code
         # now actually execute the code object
         self.runcode(code)

This means that if you modify runcode(), when you catch an exception you can 
use the self.code_to_run_src attribute to get the source which was being run, 
and do whatever you want with it.  Note that the new runcode() at the end has 
a bit of flushing to do, which your modified version will also need to do:

...
         except:
             self.showtraceback()
         else:
             if code.softspace(sys.stdout, 0):
                 print
         # Flush out code object which has been run (and source)
         self.code_to_run = None
         self.code_to_run_src = ''


I still think you'll have to rebind runcode() to your own method.  I don't see 
an easy way to make this more flexible, without jumping through convoluted 
contortions.  It's technically possible to store an arbitrary list of 
additional exceptions and handlers.  But I think at this point, rather than 
trying to parametrize every possible combination in the universe, it's simply 
cleaner for you to override one method and do exactly what you want.  This 
part of the code rarely changes, so I think the risk for you is really minimal.

[... deleted big rant about how hard this was ...]

Scratch the previous paragraph,  I just realized this was rather easy to do.

It's funny how python does these things to you. You start arguing how 
something is not a good idea, because it's too complicated to do.  And when 
you make a little test to show why it's so, you simply end up solving the 
problem.  And it takes longer to argue about why its' hard, than to
actually implement a working solution.  No wonder we get hooked on this 
language, try doing the same thing in C, Perl, or just about anything else.

In case anyone is curious, here's a simple example (outside of ipython):

import sys

def myhandler(etype,value,tb):
     print '*** my handler ***'
     print 'etype:',etype
     print 'value:',value
     print 'tb:   ',tb
     print

def customcatch(src,exc,handler):
     try:
         exec src
     except exc:
         etype,value,tb = sys.exc_info()
         handler(etype,value,tb)

customcatch('range(3)[10]',(IndexError,ZeroDivisionError),myhandler)
customcatch('1/0',(IndexError,ZeroDivisionError),myhandler)
customcatch('1/0',(ZeroDivisionError,),myhandler)

# this won't be caught:
customcatch('{1:2}[10]',(IndexError,ZeroDivisionError),myhandler)

You can run the above, even with normal python, to see the effect.

Following this, IPython now has a facility for custom exception hadling. 
IPython can take a list of custom exceptions and a handler, and call that 
handler if any exception in the list matches.  Note that this can only be done 
for a single handler, because I can't (short of doing dynamic code generation, 
which I don't want to get into)  have multiple except clauses.  So your one 
handler would get the exception info for _any_ of the matched exceptions, and 
it would have to deal with them.

Please see the set_custom_exc method for details.  You can call it at runtime 
anywhere (__IPYTHON__.set_custom_exc(...)), or you can pass the class 
constructor the relevant information, via the

   custom_exceptions=((),None)

parameter.  This param has the list of custom exceptions and their handler 
(None selects an internal dummy default).  Here's what you get with

   custom_exceptions=((ZeroDivisionError,IndexError),None) :

In [1]: x=99

In [2]: y=0

In [3]: x/y
*** Simple custom exception handler ***
Exception type : exceptions.ZeroDivisionError
Exception value: integer division or modulo by zero
Traceback      : <traceback object at 0x40379cfc>
Source code    : x/y

In [4]: a=range(3)

In [5]: a[4]
*** Simple custom exception handler ***
Exception type : exceptions.IndexError
Exception value: list index out of range
Traceback      : <traceback object at 0x40379e14>
Source code    : a[4]


This system allows you to do basically anything you want, as far as exception 
reporting and control is concerned.  This is in CVS, please let me know how it 
goes.

>    2.
> 
>       Add some new matchers for completion. As *PyMAD* uses remote objects,
>       completion only shows the client Pyro proxy. So we added a new matcher by
>       adding a IPython.iplib.MagicCompleter.proxy_matches() method, and insert
>       this matcher in ipshell.IP.Completer.matchers list. The new matcher get
>       the object (from the text param), call a special method on this object
>       which returns all available attributes (in fact, only these we want to
>       show to the user). Give the possibility to return all matchers, or only
>       the no None first. Here is the code used:

Done.  A new rc parameter, readline_merge_completions, controls this (true by 
default).

New completers can now be added by using the __IPYTHON__.set_custom_completer 
method.  Plase see its docstring for details.

>    3.
> 
>       In the same way as matchers, get the docstring from the remote object
>       instead of the client one when using 'object?' syntaxe. This could be done
>       on the same idea: calling a special method on the object, method returning
>       the doc of our remote object).

OK, this is easy.  Just make sure that the objects IPython sees have a 
.getdoc() method; I've added a call to that into my object introspection code. 
  Have a look at inspect.getdoc, you can probably have your .getdoc method be 
like that, since it does a good job of cleaning up indentation for the 
docstring.  Your method should conform to the same API as inspect.getdoc: 
return None if nothing is found, return a string (formatted to your liking) 
otherwise.

In fact, the change is already in CVS, so feel free to play with it.  See 
OInspect.py's getdoc() function for details.

>    4.
> 
>       New exception handler. Here, the idea is to be able to present different
>       kind of exceptions in different ways. Some will only print a simple
>       message, some others will print the entire traceback (maybe a modified
>       traceback).

Well, there are already three exception handlers in ipython, switchable at 
runtime and init time via xmode.  If you need more flexibility, the first step 
would be to simply subclass AutoFormattedTB from ultraTB.py.  This is what 
IPython uses for its exception handler:

         # TraceBack handlers:
         # Need two, one for syntax errors and one for other exceptions.
         self.SyntaxTB = ultraTB.ListTB(color_scheme='NoColor')
         # This one is initialized with an offset, meaning we always want to
         # remove the topmost item in the traceback, which is our own internal
         # code. Valid modes: ['Plain','Context','Verbose']
         self.InteractiveTB = ultraTB.AutoFormattedTB(mode = 'Plain',
                                                      color_scheme='NoColor',
                                                      tb_offset = 1)

You could then reassign self.InteractiveTB to be an instance of your modified 
one (you may want to also modify the convenience %xmode magic to recognize 
your new modes).  This should give you full control over exception reporting. 
    Once you are satisfied with a clean design, we can look into whether some 
of this flexibility can be put into a decent API, but for now I think it's 
easier if you just subclass and play with whatever you want.

Alternatively, with the custom exception handler mechanism (see above), you 
can just catch specifically the exceptions you'd like to treat differently and 
deal with them directly.  Which of the two approaches you take will depend 
exactly on what you want to do.

>    7.
> 
>       Access to the command-line interpreter, to have *IPython* interprets code
>       has if it was entered through keyboard (ie make difference between magic
>       commands and normal python code).

Why can't you just use the push() method?  That's pretty much what it does, 
see its docstring.  If this is not enough, you'll need to clarify to me 
exactly why not, and what else you'd need.


OK, that's quite a lot.  All of this is in CVS, please test it extensively and 
let me know how it all goes.

Regards,

f