[Tutor] Interacting with stderr

[Cameron Simpson]

On 30Aug2014 22:32, Crush <crushed26 at gmail.com> wrote:
>Thank you Allan for your criticism. Please see the below changes. As far as the embedded loops, I know no other way to achieve the same out come.

Regrettably I've lost Alan's email on that (I had it:-).

It looks like you've taken he concern over nested use of p.stderr too 
literally. It loops like you avoid using the same stderr by restarting 
"test_loop.sh" in the inner loops and reparsing their error outputs. I'm fairly 
sure that is not what Alan intended and also not what you want to be doing.  
Please explain your thinking here.

Regarding nested loops on the same iterable:

It is ok to nested use of an iterator, with some caution. The typical use case 
is some sort of nesting parse of the output. For example, if you were looking 
for the word "Segmentation", and then intending to act specially on all 
following lines until some marker. Another gleaming example that springs to 
mind would be gathering up python stack traces (multiline outputs) in some 
error log.

You'd fairly naturally end up with code a bit like this:

   for line in p.stderr:
     if "Segemntation" in line:
       # ok, found the start line
       for inner_line in p.stderr:
         if "end segemntation marker" in inner_line:
           break
         ... do something with inner_line

That essentially has the effect that all the lines from "Segmentation" to "end 
segemntation marker" are processed specially by the inner loop, and after you 
see the end marker you return to the outer loop, looking for the next occurence 
of "Segmentation".

There is really only one pitfall with this, and that is the possibility that 
the end marker line is itself important.

Imagine you had two inner loops, one for the "Segmentation" section and another 
to handle some other section, like this:

   for line in p.stderr:
     if "Segemntation" in line:
       # ok, found the start line
       ... do the "Segmentation" section ...
     elif "other-section" in line:
       # ok, found the start line
       ... do the "other-section" section ...
     else:
       # boring line, report it or discard it etc

Simple and straight forward, yes? It will nicely handly input like this:

    blah
    blah
    begin Segmentation
    >> stuff
    >> more stuff
    end segmentation marker
    blah
    blah
    begin other-section
    >> other stuff
    >> even more other stuff
    end the other section
    blah

However, consider the case where the end marker for the "Segmentation" section 
is not some special line generated by the "Segmentation" event, but simply some 
line that is not a match. Like the beginning of an "other-section" section. Eg:

    blah
    blah
    begin Segmentation
    >> stuff
    >> more stuff
    begin other-section
    >> other stuff
    >> even more other stuff
    blah

With input like that, the "end of section" line is itself an important line 
that needs to be considered. But the loop as written "consumes" the marker 
line, and it is not available for recognition on the next outer loop pass 
(because that fetches the next line).

This is where nested loops on the same iterable can have an issue. There are 
several things you can do about it depending on your code and your preferences.  

>def kill_proc(process1, process2):
>   i = psutil.Popen(["ps", "cax"], stdout=PIPE)
>   out, err = i.communicate()
>   for proc in psutil.process_iter():
>       if proc.name == process1 and process2:
>           proc.kill()

This function won't work as you intend. Specificly, the "if" test is wrong. You 
want to change:

   if proc.name == process1 and process2:

into:

   if proc.name == process1 and or proc.name == process2:

or perhaps:

   if proc.name in (process1, process2):

and your psutil.Popen call looks... confused. Popen is a subprocess function, 
not part of psutil (I think). And you don't use either "out" or "err" after you 
collect them, so why use .communicate?

Also, this will kill _every_ instance of mbplay and avconv, not just the ones 
you started. You would be better off just killing your own.

The rest of your code's function is not clear to me. Does it work?

Cheers,
Cameron Simpson <cs at zip.com.au>