Could Emacs be rewritten in Python?

[Beni Cherniavsky]

> Patrick K. O'Brien wrote:
> > If you were crazy enough to think that you could create a program
> > along the lines of Emacs, but written in Python, how would you go
> > about it?  How would you design the domain model for files, buffers,
> > windows, and frames?  How would you allow the same level of
> > customizability?  How would you map functions (or methods or whatever)
> > to keys?  Any thoughts?
> >
> > Here is the background.  I'm working on a file editor (called
> > PyAlaMode) using wxPython and Scintilla.  It's an extension of the
> > PyCrust code base.  I've got a basic version working, and it supports
> > editing multiple files and all the usual, basic features.  But I want
> > it to be as customizable and extensible as Emacs.  In fact, I want to
> > model it after Emacs, so that elisp code could be rewritten in Python
> > and work with PyAlaMode.  To do that, I've got to support the same
> > primitives as Emacs; expose files, buffers, regions, and windows the
> > same way Emacs does; handle the Emacs keybindings and ability to remap
> > keys, the same way Emacs does, etc.
> >
> > Sound like fun?

Very.  I wanted to write an emacs since I started learning it when I
was about 14(?).  And I wanted to do it in Python since I learnt
Python (about a year ago) :-).  Don't count on me contributing,
however - I'm alwready overcommited :-(.

Carl Banks wrote on 2003-04-06:

> What I recommend is to read through the Emacs Lisp manual and figure
> out the functions you'll want to support.

Yes, `info elisp` is very good to read for this; `info emacs` contains
nothing important.  Here are some more reading pointers:

- http://www.cliki.net/CL-Emacs
- http://www.emacswiki.org/cgi-bin/wiki.pl?WishList
- http://xemacs.org/Architecting-XEmacs/index.html
- http://www.gnu.org/software/emacs/emacs-paper.html
- etc/OTHER.EMACSES in the distro (join the lines):
  http://savannah.gnu.org/cgi-bin/viewcvs/*checkout*/emacs/
  emacs/etc/OTHER.EMACSES?rev=HEAD&content-type=text/plain

> Obviously, you'll want to make each type of object (buffer, windows,
> frames, positions, markers, keymaps, etc.) a Python class.  When you
> read through the manual, take the lisp functions and assign them as
> method of the appropriate class. (For now, don't *write* the methods
> yet; just prototype them.)  This builds a set of interfaces that the
> classes should follow.  The point of this is to give you an idea of
> exactly what the different objects in Emacs do, and how they
> interact.
>
> Don't be too slavish in trying to match the exact behavior of the lisp
> functions, and be careful to assign functions to the class they really
> belong to.  Don't assign a function to the keymap class just because
> it's in the keymap section.
>
Good advices.

> As an example, take the section on buffers.  First function is
> "current-buffer".  What class should it belong to?  Certainly not
> Buffer.  The current buffer is a global property, so "current_buffer"
> should be either a global function, or a method of an application
> context class.
>
Technically I believe it's the current buffer of the current window of
the current frame.  Convenience functions are needed of course.

> Next three functions are "save-current-buffer", "with-current-buffer",
> and "with-temp-buffer".  These are actually not functions, but special
> forms and macros.  Python has no analogue to these (barring some
> obscure metaclass tricks).  Depending on the form, it might be useful
> to have a corresponding method in Python, or it might be better to
> think about a more Pythonic way to do it.  I haven't thought about it
> too much, but off hand, I'd say I'd choose the latter in this case.
> In fact, these seem to be mostly a convenience, so I wouldn't bother
> to include them.
>
This is not a tiny issue trivial to give up.  This kind of macros is
the heart of elisp programming.  The normal way to e.g. find another
place in the buffer is to use functions that actually *move the point*
and record the new position - all this inside `save-excursions'.
Ditto for other tasks.  Elisp has a very imperative, plumbing style.
It's very common for functions to actually change things.

The primitives for building other functions are in part the promitives
the user itself uses when editing - or thin layers upon them.  This
also fits very well with the idea of "interactive functions", which I
think is a great idea.  When you need to combine them in other ways
something else you have all the save- and with- macros that allow you
to plumb these destructive primitives together.

> Next function is "buffer-name".  This obviously belongs in the Buffer
> class.  I'd just call it "name".  Perhaps you want to retain the exact
> name, though.  That's up to you.
>
> Next function is "rename-buffer".  This should belong in the Buffer
> class--however, in Emacs, it only operates on the current buffer.  I
> think that's a design flaw; I would fix it.  I would make "rename" a
> method of Buffer.
>
> You might want to also include global "buffer_name" and
> "rename_buffer" functions that operate on the current buffer.
>
> Next function is "get-buffer".  This should be global, because the
> list of buffers is global.
>
> And so on.
>
These look good.

> Understand what I'm saying?  Eventually, you'll have a nice list of
> classes waiting to be fleshed out.  It should be straightforward to do
> it.
>
> The next thing you should do it refactor your original PyAlaMode code
> to use these classes.  Don't add functionality.  Just implement what
> you need to get back to the original behavior of PyAlaMode, but using
> the new classes.
>
> You said the keymaps aren't editable?  Add one keymap.  Don't worry
> about adding functions to edit keymaps yet.  Say you currently use a
> string to hold the buffer.  Change it to use a Buffer object.
>
> Once you've done all this, then it should be straightforward to fill
> out the interfaces and add features.
>
Sound advices.

> There is one potentially disasterous pitfall you should be aware of.
> Emacs does all kinds of strange things with namespaces.  It uses
> dynamic binding, unlike Python, which uses lexical binding.

This in itself is a minor language issue.  It's true that sometimes
temporarily changing some settings to affect called functions is nice
but it should be easy to do it otherwise.

> Emacs has one monolithic global namespace, as opposed to Python,
> which has an organized infrastructure of "global" namespaces, that
> is, modules.

Again, minor issue.  Emacs libraries are well-disciplined to use
consistent prefixes.  Python should win here a little.

> Also, Emacs has weird namespaces like buffer local variables.  One
> thing you'll have to do is think about how to sort all these
> namespace issues out.
>
This is a tremendously important point.  Think of this before you
think anything else.  Emacs also has a some frame-local and I think
even terminal-local variables nowdays.  It didn't always have them.
Initially there were only buffer-local variables.  The point I'm
driving at is that these were intorduced *without* most code knowing
about them.  Any variable can be become buffer-local.  If you make
thme first global and then rethink them as attributes of a buffer
object, you will need to change all the code - and still one day you
will need frame-local...

If you want ot just mirror emacs, have a special proxie object
containing everything that would be a variable in emacs as attributes.
Then you can invent new kinds of localities without changing code.

The lesson is that emacs is full of quick-and-dirty tricks but there
are good reason they are there.  It's carefully designed to allow
*unforseen* customizations.  That's why there are hooks all over it,
something rare in applications that are thought out from before.
Don't fall into the pitfall of assuming that you can predict the
future customization directions completely.  When choosing good
modular structure, don't do it to minimize the code you already have
but to maximize the flexibility for code nobody thought of yet.

[Excuse my lecture tone - I didn't intend it...]

> It seems to me that a lot of these issues won't matter much if you
> have a good and consistent strategy for organizing stuff and passing
> parameters around.
>
They matter a lot; that's why you must find a good strategy <wink>.

-- 
Beni Cherniavsky <cben at tx.technion.ac.il>

When I grow up I want to write an Emacs ;-)