[IPython-dev] History

[Hans Meine]

Am Mittwoch, 16. Februar 2011, um 23:10:27 schrieb Thomas Kluyver:
> I'd like to propose simplifying this somewhat. Specifically, I think it
> makes sense to store history on disk just once. I suggest an SQLite
> database.

+0.9
(it will become a little bit harder to directly access the history, compared 
to the plain text files at least)

> - We keep the session lists of raw and translated history as they are.
> - Each command is stored instantly, so we do away with the need for an
> autosave timer thread. A crash at any stage should leave your entire
> history intact up to the last command completed.

Why "completed"?  I would assume that a crashing command will still appear in 
the DB?!

> - We store only raw history on disk (I think raw history is what we're
> looking for 90% of the time). If we want translated history, we redo the
> translation on the fly (this should require minimal computation, unless
> I've missed something).

Not sure about this one - sounds fragile, I guess there was a reason to store 
the processed commands; what about processing which depends on currently 
active aliases etc.?  My fear is that this will become difficult to recompute.

OTOH, most of the time, the two commands are absolutely equal.  What about two 
DB columns, but leaving the processed command empty if it is equal to the raw 
command?  This could save a lot of space, and it would be trivial to create a 
view which provides the two full columns.

> - History is indexed by session number and prompt number.

Sounds sensible.

> This provides a
> sensible behaviour if we have two IPython shells open together - the second
> one to be opened will be the latter session (and will be able to access
> commands entered in the other session as soon as they are completed).

I would still expect only local commands to appear in the normal readline 
history, and in the default %history output.  IMO there should be optional 
modes (something along %history --all) which list the other shells' history.

> - For magic commands, accessing a line from a previous session could look
> like "-1#9" (9th line of immediately previous session).

Hmm, what about concurrent sessions?  (Versus "previous" sessions which have 
been quit.)

> - On starting IPython, we load the last (~40 lines/~2 sessions) from the
> database into readline history.

Are you kidding me?  One session equals roughly 1400 lines for me. :-)

That's a good question, how much history should be available from "previous" 
sessions.

> Thoughts?

You wrote "will be able to access commands entered in the other session as 
soon as they are completed" - does this require ipython to constantly read 
from the DB?  No, obviously there should be a history API that directly 
operates on the database.

Only for readline, we would be reading some lines from the DB initially, 
right?

HTH,
  Hans