[Matrix-SIG] Interactive Shell (was re: advocacy)

[Janko Hauser]

Actually for personal use I have build an interactiv shell, which does 
some of these things. If I think about the missing parts I see that it 
needs a rewrite, but it is possible. Here an example of a
session.

@porto >psh
Python 1.5.2a2 (#2, Oct 26 1998, 22:21:28)  [GCC 2.7.2.3] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
(interactiv_shell)
>>> len(dir()); dir()[:10]
276
['ArrayType', 'BuiltinFunctionType', 'BuiltinMethodType', 'ClassType', 
'CodeType', 'Complex', 'Complex0', 'Complex16', 'Complex32',
'Complex64']
>>> who() # no locally defined variable
>>> a=a # here I hit TAB for expansion in local namespace
abs                arange             argsort            arrayrange
absolute           arccos             array              arraytype
add                arcsin             array2string       asarray
alltrue            arctan             array_constructor  assert
and                arctan2            array_repr         
animate            argmax             array_set          
apply              argmin             array_str          
>>> a=array([1,2,3])
>>> ?take
** doc for take **
take(a, indices, axis=0).  Selects the elements in indices from array
a along the given axis.
>>> !ls
many_nice_files.py
>>> shell.log='log_of_session.log' # I start logging input
>>> a=span(-pi,pi,200)
>>> b=cos(a)
>>> !cat log_of_session.log
a=span(-pi,pi,200)
b=cos(a)
>>> shell.log='' # I end logging of input
>>> b[:5] 
array([-1.        , -0.99950159, -0.99800685, -0.99551728,
-0.99203536])
>>> !cat log_of_session.log
a=span(-pi,pi,200)
b=cos(a)
shell.log=''    # this should not be there
>>> who() # look for local variables
a               <type 'array'>
b               <type 'array'>
>>>Ctrl-D # currently no quit but that's easy :-)

Together with commandline history from the readlinemodule this is
quite nice, but not perfect. No output logging for example.

But one should think about some design decisions. I also think emacs
is not a general answer. A GUI is not easily portable and the mainloop 
interferes with most current plotting packages. I like the cmd
interface, which is the same everywhere. One can for some things popup 
a window, like a variable browser, or a help screen. I think of this
syntax

! -> shell escape also a way to edit some methods
? -> help for object needs to look for different sources
@ -> interpreter relevant commands, possibly without parameters (no
braces)

For me it's important that all the logged lines are valid python, so
that one can right start to edit them, to have a script. So the output 
if captured needs to go elsewhere. Also the interpreter relevant
commands are escaped by the @. So it should be @who. Some funtions are 
needed for simple path, file manipulation and to change the current
directory. I don't want to introduce to much new commands with lots of 
magic, the hole system should be close to Python scripts. If one wants 
to use python she needs to learn it to some degree. There is a need
for a special import, so that the new objects are not visible by
who(). So in the end, there is work to be done, but it's also possible 
to be more comfortable right now.     

In the end I want to say, that I'm in favour for a scientific
workbench. 

__Janko
        
Perry Greenfield writes:
 > To which I would add:
 > 
 > 6) Weak interactive environment: It is not always appreciated by 
 >    non-scientists that interactive mode is where most time is spent
 >    looking at data. To that end, the interactive mode needs some conveniences
 >    that are currently lacking. Such as:
 >    - command (and output) logging.
 >    - easy system escapes (e.g., !ls in IRAF, $ls in IDL)
 >    - as trivial as it sounds, an intuitive exit command (like 'exit' or 'quit')
 >    - an easy way of re-executing moderately long command sequences or blocks
 >      without using an editor (e.g., you invoke a loop of several statements
 >      repeatedly because you need to tweak some parameters each time).
 >    - utility functions to start up help, or examine the namespace for
 >      existing data elements (much like IDL does)

 >    To these one can raise one more issue. I sense that Python gurus consider
 >    IDL users to be programming adverse, but in the spectrum of astronomical
 >    users, those that use IDL tend to be the ones that are most interested 
 >    in programming. We (STScI) also must concern ourselves with the other
 >    half (or whatever) that find even IDL too intimidating and are much more
 >    comfortable with the more shell-like environment of IRAF. There are a 
 >    lot of users that would not want to have to quote all strings or put
 >    parenthesis around all argument lists. This doesn't have a simple
 >    solution but I'm pretty sure that ignoring this issue will preclude
 >    use of Python by many astronomers (I won't speak for other fields).
 > 7) Image display convenient for astronomers (either by saoimage or ximtool
 >    lookalikes or connections to either of those so that images could be
 >    displayed on those tools.   
 > 
 > After such a list some might wonder why we even consider using Python at
 > all. Essentially the existing situation has even more serious drawbacks
 > that we are trying to escape.
 > 
 > Should our initial efforts succeed then we need solutions to the above
 > issues. We make our software available to a wide community for a number
 > of different platforms so it is simply not a matter of solving these
 > problems for our own organization. If what we deliver is too hard to 
 > install or is to hard to use or understand in the wider astronomical
 > community it will be a failure (I'm sure some would argue that existing
 > IRAF-based software falls into that category). So it is essential, for
 > example, that there is a standard distribution and installation mechanism.
 > Fortunately, there appears to be serious interest in solving this for
 > the whole Python community. For our initial plans, these issues do not
 > all have to be solved immediately, but we have to see a way to deal with
 > them eventually.
 > 
 > Hopefully the FITS issue will be solved soon--I'm eagerly looking forward to
 > Paul's implementation. (And make that ~5 FITS implementations; I did a 
 > primitive one for some test programs we were doing here). There is reason
 > to believe that documentation will improve, and we can augment it where 
 > necessary. Numerical utilities can be developed as needed. It doesn't
 > seem that one giant effort is needed there.
 > 
 > The plotting situation seems to be somewhat muddled and one of my more 
 > serious concerns. I'd rather not spend the effort to develop a package
 > from scratch. There a number of graphics packages out there but each
 > has a shortcoming or two that makes us reluctant to use it. Our need
 > is primarily for simple 2-d plotting tasks (x-y, contour, surface, etc.)
 > but that has enough control to produce publication quality output.
 > While we don't currently support Windows  platforms, we don't want to
 > rule that out. Nor can we distribute commercial software. We would 
 > prefer it to have an object oriented interface as well. This appears
 > to rule out most of what is out there. PyGIST is not available on 
 > Windows, DISTLIN is commercial, GTK appears only to provide plotting
 > primitives and doesn't run on Windows, VTK is volume rendering based
 > and has little for 2-d plotting...about the only thing available now
 > appears to be pgplot.) For the moment, I'm waiting to see what David
 > Ascher produces; it sounds promising. Complicating our decisions is
 > trying to decide how Java and JPython will fit into our future. We prefer
 > a solution that will run in JPython as well (It almost sounds like we
 > constraining ourselves out of a solution!)
 > 
 > We are working on adding some conveniences to the interpreter by making
 > our own wrapper. I'm not Guido's new IDLE environment will do it for us.
 > I haven't had a chance to take a look at it yet.
 > 
 > If we do go with Python it does mean that there will be some institutional
 > support to help supply what is missing in the above areas.
 > 
 > Perry Greenfield
 > Space Telescope Science Institute
 > Science Software Group
 >