[Tutor] Retrieve data from log file

[Peter Otten]

Abe Miranda wrote:

> Hi there,
> 
> I'm sorry to ask this, I tried and example that was posted in 2009 but I
> couldn't make it work. I'm trying to parse a log file and retrieve what I
> just need.
> 
> Hope you can help me with this, this is part of the log file:
> It repeats itself for every transaction and I need to extract from this
> lines:
> 
> [2011-10-12 21:02:43:383] DEBUG  [BaseQualitasDaoWS        :  86]
> [http-80-1][clientThread-Thread-3] -wscall - hours:0.0 minutes:0.0
> seconds:3.0 &
> [2011-10-12 21:02:43:462] DEBUG  [RestController           :  81]
> [http-80-1][clientThread-Thread-3] - hours:0.0 minutes:0.0 seconds:16.0&
> 
> this information:
> 
> 2011-10-12 21:02:43:383   clientThread-Thread-3   hours:0.0 minutes:0.0
> seconds:3.0
> 2011-10-12 21:02:43:462   clientThread-Thread-3   hours:0.0 minutes:0.0
> seconds:16.0
> 
> Can this be done?

I'd start with a few simple tests like

with open("test.log") as instream:
    for line in instream:
        if line.startswith("["):
            parts = line.replace("[", "]").split("]")
            duration = parts[8].strip("& \n")
            if "hours" in duration and "minutes" in duration:
                duration = duration[duration.index("hours"):]
                time = parts[1]
                thread = parts[7]
                print "%s\t%s\t%s" % (time, thread, duration)

and see if you get false positives/negatives. Refine as necessary.