storing test logs under /var/log/

[Cameron Simpson]

On 03Dec2015 12:20, Ganesh Pal <ganesh1pal at gmail.com> wrote:
>I would need few tips  from your past  experiences on  how  to store
>the test logs
>My requirement is to capture log under /var/log/  directory every time
>the test is run .

Normally /var/log itself is only writable by root. Unless you are running your 
tests as root (_not_ recommended unless you're testing something quite 
privileged) the convention is either to create a log directory in the home 
directory of the user running the tests (though not if running as root), *or* 
to create a subdirectory in /var/log named after your purpose, eg 
/var/log/test-results.

>The test will create one small log files  which are
>around 1KB in size .

Fine, unless you're running some poor filesystem (unlikely these days).

>Here is how I plan to approach this , create directory based on
>current timesamp and store the logs in it .

Sounds reasonable to me, but definitiely inside a top level directory _below_ 
/var/log.

>Sample code :
>
>time_now = datetime.datetime.now().strftime('%Y-%m-%d_%H-%M-%S') + "/"
>LOG_DIR = ""  +  time_now

This is a little weird in a few ways.

Firstly, UPPER_CASE names are normally reserved for what would be called 
"constants" in languages like C: top level global names for program wide 
defaults. You don't use them for working, changable, names like the computed 
name of your log directory. I'd be saying something like this:

  # near the top of your program
  TOP_LOG_DIR = '/var/log/test-results'

  # lower down when computing the subdirectory for this test run
  time_now = datetime.datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
  results_dir = os.path.join(TOP_LOG_DIR, time_now)

Notice also that this example code does not mention the '/'; on UNIX of course 
that is the path separator, and /var/log is a UNIX thing, but generally you 
should not unnecessarily put in system depending stuff like '/' (vs '\' on say 
Windows) in the middle of your code. So use os.path.join to construct 
pathnames.

> try:
>         retcode = os.makedirs(LOG_DIR)
>         if retcode:
>             raise Exception(
>                     "Faild to create log directory. mkdir %s failed !!!"
>                             % LOG_DIR)
>    except Exception, e:
>        sys.exit("Failed to create log directory...Exiting !!!")
>
>1.  Do  I need to add the cleanup code  to remove the  log directory
>say  /var/log/test_log/2015-11-25_04-07-48/   , because we might have
>many more directories like this when test are run multiple times , Iam
>avoiding because  the test will be  run
> 2/3 times max and file sizes are also very small

I would also avoid it. I would leave that as an entirely separate task for 
another program (quite possibly a shell script whose sole purpose it to tidy up 
things like that). That way a user can safel;y run your test program _without_ 
being in fear that it my, unasked, delete a whole bunch of earlier tests data.

Having your test program "clean up" is effectively wiring in a policy decision 
into your program, where users cannot avoid or change it. All programs 
implement some policy (arbitrary decisions about things beyond their core 
purpose, such as tidyup of data they did not create), but that should be 
minimised; that way the users can decide on policy themselves.

>2. Any better suggestion for my use case.

I would change your try/except code above a little:

os.makedirs does not return zero/nonzero on success/failure like a C library 
function, it raises an exception. So remove your if statement, changing this:

  retcode = os.makedirs(LOG_DIR)
  if retcode:
    raise Exception(
      "Faild to create log directory. mkdir %s failed !!!" % LOG_DIR)

into:

  os.makedirs(results_dir)  # see earlier code

Also, you discard the value "e" of the exception. It is almost always best to 
report exceptions tso that the user can figure out what went wrong, not merely 
to know that something (unidentified) went wrong.

And because os.makedirs raises an except, and the default behaviour of a python 
program (without a try/except) is the abort with a nonzero exit and _also_ 
helpfully report the whole exception and the code which caused it, you could 
completely discard your whole try/except!

Finally. sys.exit accepts an integer, not a string.

Cheers,
Cameron Simpson <cs at zip.com.au>