piping with subprocess

[Rick Dooling]

On Saturday, February 1, 2014 7:54:34 AM UTC-6, Rick Dooling wrote:
> On Saturday, February 1, 2014 6:54:09 AM UTC-6, Peter Otten wrote:
> 
> > Rick Dooling wrote:
> 
> > 
> 
> > 
> 
> > 
> 
> > > I spent half a day trying to convert this bash script (on Mac)
> 
> > 
> 
> > > 
> 
> > 
> 
> > > textutil -convert html $1 -stdout | pandoc -f html -t markdown -o $2
> 
> > 
> 
> > > 
> 
> > 
> 
> > > into Python using subprocess pipes.
> 
> > 
> 
> > > 
> 
> > 
> 
> > > It works if I save the above into a shell script called convert.sh and
> 
> > 
> 
> > > then do
> 
> > 
> 
> > > 
> 
> > 
> 
> > > subprocess.check_call(["convert.sh", file, markdown_file])
> 
> > 
> 
> > > 
> 
> > 
> 
> > > where file and markdown_file are variables.
> 
> > 
> 
> > > 
> 
> > 
> 
> > > But otherwise my piping attempts fail.
> 
> > 
> 
> > 
> 
> > 
> 
> > It is always a good idea to post your "best effort" failed attempt, if only 
> 
> > 
> 
> > to give us an idea of your level of expertise.
> 
> > 
> 
> > 
> 
> > 
> 
> > > Could someone show me how to pipe in subprocess. Yes, I've read the doc,
> 
> > 
> 
> > > especially
> 
> > 
> 
> > > 
> 
> > 
> 
> > > http://docs.python.org/2/library/subprocess.html#replacing-shell-pipeline
> 
> > 
> 
> > > 
> 
> > 
> 
> > > But I'm a feeble hobbyist, not a computer scientist.
> 
> > 
> 
> > 
> 
> > 
> 
> > Try to convert the example from the above page
> 
> > 
> 
> > 
> 
> > 
> 
> > """
> 
> > 
> 
> > output=`dmesg | grep hda`
> 
> > 
> 
> > # becomes
> 
> > 
> 
> > p1 = Popen(["dmesg"], stdout=PIPE)
> 
> > 
> 
> > p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
> 
> > 
> 
> > p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
> 
> > 
> 
> > output = p2.communicate()[0]
> 
> > 
> 
> > """
> 
> > 
> 
> > 
> 
> > 
> 
> > to your usecase. Namely, replace
> 
> > 
> 
> > 
> 
> > 
> 
> > ["dmesg"] --> ["textutil", "-convert", "html", infile, "-stdout"]
> 
> > 
> 
> > ["grep", "hda"] --> ["pandoc", "-f", "html", "-t", "marktown", "-o",
> 
> > 
> 
> >                      outfile]
> 
> > 
> 
> > 
> 
> > 
> 
> > Don't forget to set
> 
> > 
> 
> > 
> 
> > 
> 
> > infile = ... 
> 
> > 
> 
> > outfile = ... 
> 
> > 
> 
> > 
> 
> > 
> 
> > to filenames (with absolute paths, to avoid one source of error).
> 
> > 
> 
> > If that doesn't work post the code you wrote along with the error messages.
> 
> 
> 
> p1 = subprocess.Popen(["textutil", "-convert", "html", file], stdout=subprocess.PIPE)
> 
> p2 = subprocess.check_call(["pandoc", "-f", "html", "-t", "markdown", "-o", markdown_file], stdin=p1.stdout, stdout=subprocess.PIPE)
> 
> p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
> 
> output = p2.communicate()[0]
> 
> 
> 
> Errors
> 
> 
> 
> Traceback (most recent call last):
> 
>   File "/Users/me/Python/any2pandoc.py", line 70, in <module>
> 
>     convert_word_file(file, markdown_file)
> 
>   File "/Users/me/Python/any2pandoc.py", line 59, in convert_word_file
> 
>     output = p2.communicate()[0]
> 
> AttributeError: 'int' object has no attribute 'communicate'
> 
> 
> 
> I get a markdown_file created but it's empty.
> 
> 
> 
> Thanks,
> 
> 
> 
> RD
> 
> 
> 
> ps - Daniel's works fine but I still don't learn to pipe :)

Okay, sorry. I fixed that obvious goof

    p1 = subprocess.Popen(["textutil", "-convert", "html", file], stdout=subprocess.PIPE)
    p2 = subprocess.Popen(["pandoc", "-f", "html", "-t", "markdown", "-o", markdown_file], stdin=p1.stdout, stdout=subprocess.PIPE)
    p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
    output = p2.communicate()[0]

Now I get no errors, but I still get a blank markdown file.