Only getting the first 6 lines

[Cecil Westerhof]

On Friday  2 Oct 2015 00:50 CEST, Cameron Simpson wrote:

> On 01Oct2015 23:58, Cecil Westerhof <Cecil at decebal.nl> wrote:
>> I want to get the first 6 lines of ps output. For this I use:
>> ========================================================================
>> from subprocess import check_output
>>
>> ps_command = ('ps', '-eo',
>> 'user,pid,pcpu,pmem,stat,start,time,cmd', '--sort') message =
>> '\n'.join(check_output(ps_command +
>> ('-%cpu',)).decode("utf-8").splitlines()[0:6])
>> ========================================================================
>>
>> It works, but does not look very efficient. Is there a better way
>> to do this?
>
> It depends what you mean by inefficient. I'm presuming you mean that
> this:
>
> - sucks in all the output of ps instead of just the first 6 lines
>
> - sucks all the output into memory instead of just some of it
>
> - does more in-memory work in splitlines()

Yes.


> - needs ps to run to completion instead of just long enough to print
>   6 lines

No, because of the sort, ps has to generate all lines.
>
> You could just read six lines of output from ps. Something like this
> (untested):
>
>
> lines = []
> for lineno, line in \
> enumerate(Popen(ps_command + ('-%cpu',),
> stdin=NULL, stdout=PIPE).stdout,
> 1):
> lines.append(line.decode("utf-8"))
> if lineno >= 6:
> break


I wrote the following general function for it:
========================================================================
def log_resource_usage(command, resource_type, nr_of_lines):
    lines   = []
    process = Popen(command, stdin = None, stdout = PIPE, stderr = None)
    for lineno, line in enumerate(process.stdout, 1):
        lines.append(line.decode('utf-8').strip())
        if lineno >= nr_of_lines:
            break
    message = '\n'.join(lines)
    cursor.execute(insert_message, (resource_type, message))
    process.wait()
========================================================================


> This approach is important if the output of your command is large or
> slow, as it does not need to suck the whole thing into memory or to
> wait for it to finish a long procedure. With "ps" these issues are
> pretty minor; with some other programs it can be significant,
> especially if the other program _doesn't_ terminate (consider "tail
> -f" of an active log file).

Well ps generates more as 400 lines. Not a real problem, but I like to
solve problems before they become a problem. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof