[New-bugs-announce] [issue26870] Unexpected call to readline's add_history in call_readline

[Tyler Crompton]

New submission from Tyler Crompton:

I was implementing a REPL using the readline module and noticed that there are extraneous calls to readline's add_history function in call_readline[1]. This was a problem because there were some lines, that, based on their compositions, I might not want in the history. Figuring out why I was getting two entries for every 

The function call has been around ever since Python started supporting GNU Readline (first appeared in Python 1.4 or so, I believe)[2]. This behavior doesn't seem to be documented anywhere.

I can't seem to find any code that depends on a line that is read in by call_readline to be added to the history. I guess the user might rely on the interactive interpreter to use the history feature. Beyond that, I can't think of any critical purpose for it.

There are four potential workarounds:

1. Don't use the input function. Unfortunately, this is a non-solution as it prevents one from using Readline/libedit for input operations.
2. Don't use Readline/libedit. For the same reasons, this isn't a good solution.
3. Evaluate get_current_history_length() and store its result. Evaluate input(). Evaluate get_current_history_length() again. If the length changed, execute readline.remove_history_item(readline.get_current_history_length() - 1). Note that one can't assume that the length will change after a call to input, because blank lines aren't added to the history. This isn't an ideal solution for obvious reasons. It's a bit convoluted.
4. Use some clever combination of readline.get_line_buffer, tty.setcbreak, termios.tcgetattr, termios.tcsetattr, msvcrt.getwche, and try-except-finally blocks. Besides the obvious complexities in this solution, this isn't particularly platform-independent.

I think that it's fair to say that none of the above options are desirable. So let's discuss potential solutions.

1. Remove this feature from call_readline. Not only will this cause a regression in the interactive interpreter, many people rely on this behavior when using the readline module.
2. Dynamically swap histories (or readline configurations in general) between readline-capable calls to input and prompts in the interactive interpreter. This would surely be too fragile and add unnecessary overhead.
3. Document this behavior and leave the code alone. I wouldn't say that this is a solution, but it would at least help other developers that would fall in the same trap that I did.
4. Add a keyword argument to input to instruct call_readline to not add the line to the history. Personally, this seems a bit dirty.
5. Add a readline function in the readline module that doesn't rely on call_readline. Admittedly, the implementation would have to look eerily similar to call_readline, so perhaps there could be a flag on call_readline. However, that would require touching a few files that don't seem to be particularly related. But a new function might be confusing since call_readline sounds like a name that you'd give such a function.

I think that the last option would be a pretty clean change that would cause the least number of issues (if any) for existing code bases. Regardless of the implementation details, I think that this would be the best route—to add a Python function called readline to the readline module. I would imagine that this would be an easy change/addition.

I'm attaching a sample script that demonstrates the described issue.

[1]: https://github.com/python/cpython/blob/fa3fc6d78ee0ce899c9c828e796b66114400fbf0/Modules/readline.c#L1283
[2]: https://github.com/python/cpython/commit/e59c3ba80888034ef0e4341567702cd91e7ef70d

----------
components: Library (Lib)
files: readline_history.py
messages: 264360
nosy: Tyler Crompton
priority: normal
severity: normal
status: open
title: Unexpected call to readline's add_history in call_readline
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file42626/readline_history.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26870>
_______________________________________