[Tutor] automatic checking of web pages

[D-Man]

On Mon, Mar 19, 2001 at 01:42:42PM -0800, Glen Bunting wrote:
| Hi, 
| 
| I am trying to re-write a shell script that i have that checks for and
| verifies specified content on an html page.  The following is the area that
| I am having problems with:
| 
|  os.chdir('/sec/bb/ext/bb-hcheck/cfg')
|     LIST = os.listdir(os.getcwd())
|     for item in LIST:
|         #NAME = 'cut --character=5- ' + item
|         SERVER = os.popen('cut --character=5- ' + item).read()
                                               ^
Is that space supposed to be there?  The shell will separate
--character=5-  and the contents of the variable item into separate
arguments.

|         LOOK_FOR = os.popen('cut -- character=9- ' + item).read()
                                                  ^
|         #RESULTS = os.system('LYNX SERVER | grep LOOK_FOR')
|         RESULTS_TEMPLATE = 'curl --include --max-time 30 %(server)s | grep
| %(LOOK_FOR)s'
|         RESULTS = RESULTS_TEMPLATE % {'server' : SERVER,
|                                      'look_for': LOOK_FOR }
|         FINAL = os.system(RESULTS)
|         if [ RESULTS != 0 ]:
|             COLOR = 'red'
|             STATUS = 'Specified content:' +  LOOK_FOR + 'was not found'
|         else:
|             COLOR = 'green'
|             STATUS = 'Successfully found ' + LOOK_FOR
| 
| 
| running the program I get the following errors:
|  ./bb-hcheck1.py
| cut: you must specify a list of bytes, characters, or fields
| Try `cut --help' for more information.
| Traceback (innermost last):
|   File "./bb-hcheck1.py", line 59, in ?
|   File "./bb-hcheck1.py", line 34, in main
| KeyError: LOOK_FOR
| 
| Does anyone have any idea what I'm doing wrong?  Am I taking the  right
| track by trying to call the same program(curl,cut) that the shell script
| uses, or should I be doing this completely differently? 

If you get the arguments right, you can get away with using the same
external programs.  I would recommend, however, trying to use the
python library to help with this (see htmllib).  If you only want to
run the script on unix systems that has these external programs
in the path, and assume that they interpret the arguments as you
expected them to (oftentimes GNU programs have different/more
arguments than non-GNU programs,  ex  ls tar make) then it is ok to
use os.system an use them.  If you want your script to be as portable
as python is, then you should only use standard python modules (or
your own modules, of course) and not rely on external programs.

-D