[Tutor] implementing sed - termination error

[Peter Otten]

bruce wrote:

> Hi
> 
> Running a test on a linux box, with python.
> 
> Trying to do a search/replace over a file, for a given string, and
> replacing the string with a chunk of text that has multiple lines.
> 
> From the cmdline, using sed, no prob. however, implementing sed, runs
> into issues, that result in a "termination error"
> 
> The error gets thrown, due to the "\" of the newline. SO, and other
> sites have plenty to say about this, but haven't run across any soln.
> 
> The test file contains 6K lines, but, the process requires doing lots
> of search/replace operations, so I'm interested in testing this method
> to see how "fast" the overall process is.
> 
> The following psuedo code is what I've used to test. The key point
> being changing the "\n" portion to try to resolved the termination
> error.

Here's a self-contained example that demonstrates that the key change is to 
avoid shell=True. 

$ cat input.txt
foo
alpha
beta foo gamma
epsilon
foo zeta
$ sed s/foo/bar\\nbaz/g input.txt
bar
baz
alpha
beta bar
baz gamma
epsilon
bar
baz zeta
$ python3
Python 3.4.3 (default, Sep 14 2016, 12:36:27) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.call(["sed", "s/foo/bar\\nbaz/g", "input.txt"])
bar
baz                                                                                                                                                       
alpha                                                                                                                                                     
beta bar                                                                                                                                                  
baz gamma                                                                                                                                                 
epsilon                                                                                                                                                   
bar                                                                                                                                                       
baz zeta                                                                                                                                                  
0                                                                                                                                                         

Both the shell and Python require you to escape, so if you use one after the 
other you have to escape the escapes; but with only one level of escapes and 
a little luck you need not make any changes between Python and the shell.