pymacs! :-)

[François Pinard]

[Brian McErlean]

> Oh dear - oddly enough I'm currently writing exactly the same thing -
> I'd even decided to call it pymacs :-) Rather worryingly, it looks like
> I've even implemented it in a similar way.

That's why it is a good thing to announce what we are working at, if we
know we have any chance to get through, so to trigger collaboration! :-)

> > In LISP, I also gave me some more direct tools:
> > 
> >    (exec-python "PYTHON-STATEMENTS")
> >    (eval-python "PYTHON-EXPRESSION")
> >    (apply-python FUNCTION ARGUMENTS)

> I just have one function, (pymacs-eval "python-statement-or-expression").
> This evaluates either, but returns nil for a statement.  [...]  Actually,
> I think your way having seperate exec and eval may be a better idea -

So far, on my side, I found out that "python-exec" (I moved "python" as a
prefix rather than a suffix) is not really needed in practice.  The most
useful is "python-apply", yet the user almost never use it directly: these
are only used indirectly through the LISP "defun" which get installed in the
LISP space through the action of "python-import".  As for "python-eval",
I found it useful for writing quick tests, but I would not think it very
useful in real applications.

In practice on the LISP side, one does "(python-import MODULE)" and merely
use the imported functions, without paying much attention anymore if they
were written in LISP or Python.  To make thins a bit more transparent,
I'm pondering some "(autoimport ...)"  function to mimick "(autoload ...)",
unless -- this is unlikely -- I find a neat way to overload "(autoload ...)".

> > From the Python side, I may also call Emacs for help:
> >    eval_lisp("LISP-EXPRESSION")

Since then, I changed that to the simpler:

    from pymacs import lisp
    ...
    value = lisp("LISP-EXPRESSION")

In fact, not much besides `lisp' is needed from the `pymacs' module,
and Python functions not needing Emacs services do not even need `lisp'.
The rest is fairly automatic.

> Sometimes doing seemingly simple things can mean a lot of work - eg.
> (funcall (pymacs-eval "map") (pymacs-eval "lambda x:x+1") (1 2 3 4 5))

I paid attention so opaque LISP objects could be made available on the
Python side, and found back automatically when transmitted back to the
LISP side from Python.  You paid attention to the other way, and found
ways to represent non-simple Python objects on the LISP side.  So it seems
we solved complementary problems, and this is enough for me to be quite
interested in your works! :-)

I confess that I did not tackle Python opaque representations in LISP,
because I do not see how to avoid memory leaks.  If one creates a Python
object meant to be returned on the LISP side, one has to keep a reference
active to that object on the Python side, and later copies of corresponding
references could be made on the LISP side.  But then, how do one gets a clue
about when the references on the LISP side still exists or not?  Without
clues, I do not see other choices than keeping the Python object around
forever, and it might not be acceptable in long running Emacs sessions,
where a lot of Python objects could be progressively tied in this way.

On the other hand, I easily forgave myself for not doing so, as I was not
loosing so much.  I aim writing LISP extensions in the Python language,
much more than give a Python flavour to my LISP writings.  After all,
I'd rather write in Python than in LISP whenever I seek a Python flavour.
I think LISP objects like windows, buffers, markers, overlays, etc. should
be fully usable from the Python side, even if this implies a lot of automatic
synchronisation between Emacs and LISP.  I guess I reached that goal.

> > Two things are missing before I'm happy.  First, this is all too
> > fragile to errors: a Python exception should properly be seen on the
> > Emacs side, and vice-versa.

> My error handling is fairly simple really - I have a special character
> I return to python as the response to a call - ">" indicates emacs is
> makeing another call to python, "&" indicates a response to pythons call,
> and "!" indicates an error, after which I just throw a LispError() - which
> will in turn be passed to emacs using an error code.  It would be nice
> to provide some kind of backtrace up both emacs and python calls though.

Currently in my things, the Python backtrace gets displayed in the
mini-buffer (which is resized for the circumstance) *and* the LISP backtrace
is shown in the `*Backtrace*' window.  Together with the `*Pymacs*'
communication window, that makes plenty for debugging.  I made no kind of
effort to transmit the LISP backtrace on the Python side, as I do not see
a purpose for it: all debugging is done within Emacs windows anyway.

If cross-calls between LISP and Python nest deeply, an error will raise
successive exceptions alternatively on both size as things unstack, and
the diagnostic gets transmitted back and forth, slightly growing as we go.
I hope my logic is correct: these things are a bit complex to sort out
and I did not overly tested it yet.

> I have several functions on the emacs class:

> emacs.get_func(name)  - looks up (symbol-function 'name) and returns
> it, or None
> emacs.get_value(name) - just calls emacs.eval(name)
> and __getitem__       - return get_func(name) or get_value(name)

As you might guess, I had to think on similar lines, but was happy to
use `__call__' to sort out if a LISP object is meant as a function or as
a value.  Simplifying my code, I found out that the easiest avenue was to
merely shovel that problem on the LISP side, and I guess I did it cleanly.

> I considered using automatic `_' to `-' and having syntax like
> emacs.add_hook, using `__getattr__' but I didn't like the automatic
> conversion - what do you do for functions that do contain `_'?

The automatic conversion is really useful, it is well worth, so I support
it fully on both sides.  In my current Emacs here, there are only four
symbols having an underscore: `shift_jis', `shift_jis-dos', `shift_jis-mac',
and `shift_jis-unix', and they are only useful for their property lists.
Because "lisp.shift_jis" would really refer to `shift-jis', in such cases,
I would use "lisp['shift_jis']".  Automatic conversion only occurs for
the attribute notation.

> Your implementation does sound very interesting though - maybe we should
> collaborate and take the best of our respective designs?

By all means!  Deep down, these are the goal and result which attract me,
not really imposing my ideas.  A single better tool has more chance growing
roots, than two competing and lesser packages.  You understand that I would
like enjoying some ideas I had, when I dare considering them good ideas! :-)

> Currently, I had planned on releasing my version sometime next week.  
> I don't have the source with me at work, but I'll see about maybe
> posting a preliminary release sometime soon.

Do you prefer that we post our respective versions and let these influence
our respective designs, or that we seek some integrated solution through
private communications, before posting?  Or should we discuss publicly?
It depends a bit on if there are many Emacs users who have practical
interest in writing `.py'/`.pyc' files next to their `.el`/`.elc' files!

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard