[Spambayes] How to find the email from _pop3proxy.log

[Jeremy Hylton]

On Fri, 2003-06-06 at 14:50, papaDoc wrote:
> Hi,
> 
> I have this message when pop3proxy has the log file enable
> 
> > placing 1054917361-4 in corpus cache
> > Flushing 1054814303 from corpus cache
> > placing 1054826304 in corpus cache
> > Flushing 1054827795 from corpus cache
> > error: uncaptured python exception, closing channel 
> > <__main__.ServerLineReader c
> > onnected at 0xb1aff0> (email.Errors.HeaderParseError:Not a header, not 
> > a continu
> > ation: ``charset="iso-8859-1"'' 
> > [C:\Devtools\Python\PYTHON~1.2\lib\asyncore.py|p
> > 18 3990
> > 19 8807
> > 20 5457
> > 21 16741
> > 22 5040
> > .
> > UIDL
> > +OK unique-id listing follows
> > 1 2630-1049602267
> > 2 2631-1049602267
> > 3 2632-1049602267
> > 4 2633-1049602267
> > 5 2634-1049602267
> > 6 2635-1049602267
> > 7 2636-1049602267
> > 8 2637-1049602267
> > 9 2638-1049602267
> > 10 2639-1049602267
> > 11 2640-1049602267
> > 12 2641-1049602267
> > 13 2642-1049602267
> > 14 2643-1049602267
> > 15 2644-1049602267
> > 16 2645-1049602267
> > 17 2646-1049602267
> > 18 2647-1049602267
> > 19 2648-1049602267
> > 20 2649-1049602267
> > 21 2650-1049602267
> > 22 2651-1049602267
> > .
> > XSENDER 17
> > +OK
> > RETR 17
> 
> 
> How can I find the email which is causing problem  ?

I had to cope with this problem yesterday.  You can't find out the
information from the log, because the proxy doesn't log a message until
after it is parsed.  I edited the proxy source to add an extra log
output before the message is parsed.  This is just a band-aid, though,
since it logs messages twice.

I think this patch should be sufficient:

Index: pop3proxy.py
===================================================================
RCS file: /cvsroot/spambayes/spambayes/pop3proxy.py,v
retrieving revision 1.80
diff -c -r1.80 pop3proxy.py
*** pop3proxy.py	5 May 2003 01:23:07 -0000	1.80
--- pop3proxy.py	6 Jun 2003 20:38:33 -0000
***************
*** 432,437 ****
--- 432,439 ----
              # Break off the first line, which will be '+OK'.
              ok, messageText = response.split('\n', 1)
  
+             state.logFile.write(messageText)
+             state.logFile.flush()
              msg = spambayes.message.SBHeaderMessage()
              msg.setPayload(messageText)
              msg.setId(state.getNewMessageName())

I've been having a bunch of problems with the proxy, but never enough to
spend much time to fix them.  Anyone else interested in making some UI
and robustness improvements?

The chief problem I have is that if the proxy can't parse a message, it
raises an unhandled exception.  asyncore prints an error message but
doesn't close the socket.  So the client is hung until I explicitly
cancel the pop request.  Then I have to go to the POP server's web
interface, figure out which message caused the problem, and manually
remove it from the spool.  Then I can restart the proxy and the POP
client and fetch my mail.

The proxy should be a lot more robust in this situation.  I'd prefer to
see it pass through a message without scoring than have it get stuck
forever.

I'd also like to see the log opened for appending -- "a" instead of
"w".  I've made that change locally, because it doesn't make any sense
to delete the old log on restart.

I'd like to see a different UI that didn't take so long to render its
output.  When I review and score messages, it can often takes more than
a minute to render the generated HTML.  (I get more than 1000 emails a
day.)  I tend to do unsure-only training, so I waste a lot of time
clicking discard for the ham and spam messages and scrolling to the
bottom.

The other UI issue is that reviewing messages is completely decoupled
from reading messages.  I never train on ham or spam, because I'd make
too many mistakes.  I have occasionally found a misclassified message
reading mail several days after it arrived.  At that point, it's too
late to difficult to figure out how to train the proxy on it.  I've
already discarded it when I reviewed messages on the day they arrived.

Jeremy