Parsing a log file

CG cgusenetmail at hotmail.com
Sat Aug 13 17:01:41 EDT 2005


I am looking for a way to parse a simple log file to get the
information in a format that I can use.  I would like to use python,
but I am just beginning to learn how to use it.  I am not a programmer,
but have done some simple modifications and revisions of scripts.  I am
willing to attempt this on my own, if someone can point me in the right
direction (any example scripts that do similar things would be
helpful).  This doesn't have to be Python, but I need a cross-platform
solution (i.e. Perl or some other kind of script).  I just wanted to
try Python because I like the concept of it.

Here is my scenario:
I have a program that connects and disconnects to a server.  It writes
a simple log file like this:

08-13-2005  13:19:37:564 Program: CONNECTED to 'Server'
08-13-2005  15:40:08:313 Program: DISCONNECTED from 'Server'
08-13-2005  15:45:39:234 Program: CONNECTED to 'Server'
08-13-2005  15:55:18:113 Program: DISCONNECTED from 'Server'
08-13-2005  16:30:57:264 Program: CONNECTED to 'Server'
08-13-2005  16:59:46:417 Program: DISCONNECTED from 'Server'
08-13-2005  17:10:33:264 Program: CONNECTED to 'Server'
08-13-2005  18:25:26:316 Program: DISCONNECTED from 'Server'
08-13-2005  18:58:13:564 Program: CONNECTED to 'Server'
08-13-2005  19:29:10:715 Program: DISCONNECTED from 'Server'

What I basically want to do is end up with a text file that can be
easily imported into a database with a format like this (or I guess it
could be written in a SQL script form that could write directly to a
database like Mysql):

Connect_Date Connect_Time Disconnect_date Disconnect_time User
------------ ------------ --------------- --------------- -------
08-13-2005   13:19:37     08-13-2005      15:40:08        John
08-13-2005   15:45:39     08-13-2005      15:55:18        John
08-13-2005   16:30:57     08-13-2005      16:59:46        John
08-13-2005   17:10:33     08-13-2005      18:25:26        John
08-13-2005   18:58:13     08-13-2005      19:29:10        John

Here are some notes about this:
* the username would come from the log file name (i.e.
John_Connect.log)
* I don't need the fractions of seconds in the timestamps
* I only need date, time, and connect or disconnect, the other info is
not important
* If it is possible to calculate the elapsed time between Connect and
Disconnect and create a new field with that data, that would help (but
I can easily do that with SQL queries)
* This log file layout seems to be consistent
* There may not be a "disconnect" statement if the log file is read
while connected, so the next time it would have to insert the
disconnect information.  The file will be read quite regularly, so this
is very likely.
* This would eventually need to be done without intervention (maybe
every 5 minutes).

I am open to other ideas or existing programs and am flexible about the
final solution.

Thanks,
Clint




More information about the Python-list mailing list