Using subprocess to capture a progress line

[Chris Warrick]

On 10 November 2015 at 23:47, Tim Johnson <tim at akwebsoft.com> wrote:
> Using python 2.7.6 on ubuntu 14.04
> The application in question is run with bash and gnome-terminal :
>
> I've written a command-line "wrapper" for youtube-dl, executing
> youtube-dl as a subprocess.
>
> ------------------------------------------------------------------
> youtube-dl reports download progress on one line. I.E. the line is
> overwritten numerous times with no carriage return until the
> downloading is finished.
> ------------------------------------------------------------------
>
> The following code runs the youtube-dl command and reports each line
> as output by youtube-dl
> ###########
>     p = subprocess.Popen(list(args), stderr=subprocess.STDOUT,
>                          stdout=subprocess.PIPE)
>     while True:
>         line = p.stdout.readline()
>         if not line:
>             break
>         tmp = line.strip()
>         print tmp
> ###########
>
> However this method not does show the download progress _until_ the
> download is complete.

There is no \n character at the end — which means that
p.stdout.readline() cannot return. In fact, if you printed repr() of
the line you read, you would get this:

b'\r[download]  54.9% of 2.73MiB at 26.73KiB/s ETA 00:47\r[download]
55.0% of 2.73MiB at 79.33KiB/s ETA 00:15\r…snip…\r[download] 100% of
2.73MiB in 00:01\n'

The download line is implemented using \r, which is the carriage
return character (return to the first character), and then by
overwriting characters that were already printed.

The solution? There are numerous. I’ll help you by obscuring the worst one.

(1) [recommended] figure out how to make youtube_dl work as a library,
read its main file to figure out the problem. Don’t mess with
subprocess.
(2) [don’t do it] do you need to intercept the lines? If you don’t set
stderr= and stdout=, things will print just fine.
(3) [DON’T DO IT] .ernq() punenpgre ol punenpgre naq znxr n zrff.

PS. Thank you for setting a sensible Reply-To header on your messages.
Which is something the list should be doing.

-- 
Chris Warrick <https://chriswarrick.com/>
PGP: 5EAAEA16