line duplication using logging to file

[Paul McGuire]

"seb" <sebastien.thur at laposte.net> wrote in message 
news:1168329975.491514.282000 at 11g2000cwr.googlegroups.com...
> Hi, I am writing to a file some basic information using the logging
> module.  It is working but in the log file some line are printed
> several time. I had put some print debugging messages in the logging
> function (so they appear on the consile) and they are called once only.
> Obviously there is some understantding of the logging module that I am
> missing.
>
> My simple logging program (see below) is called by several processes.
> In this way I can collect the information from various sources (and not
> use the network enabled logging module)
>
> I am using python 2.4 on WinXP SP2.
>
> Do you have any idea ? Thanks in advance.
>
> Seb.
>

A quick tally of log messages by timestamp and comment gives this data:

('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
('2007-01-08 18:26:19,578', '___run____thread lance') : 2
('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
('2007-01-09 08:51:26,733', '___run____thread lance') : 2
('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1

Does this suggest anything to you?

-- Paul


(BTW, here is the pyparsing program I used to do this analysis)

from pyparsing import 
Word,nums,Combine,alphas,oneOf,Literal,SkipTo,restOfLine

# create pyparsing grammar definition for a log line
date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word(nums,exact=2)+ \
        ','+Word(nums,exact=3)
timestamp = Combine(date + ' ' + time)
severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this 
data
backslash = Literal("\\")
fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") + 
".py")
logline = ( timestamp.setResultsName("timestamp") + "-" +
            severity.setResultsName("severity") + "-" +
            fileref.setResultsName("fileref") +
            restOfLine.setResultsName("comment") )

# create list of ParseResults, with addressable log line elements by results 
name
logEntries = [ logline.parseString(line) for line in logdata ]

# tally up log lines by timestamp and comment
tallyByTimestamp = {}
for entry in logEntries:
    tallyKey = (entry.timestamp, entry.comment)
    tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1

for ts in sorted( tallyByTimestamp.items() ):
    print "%s : %d" % ts