Fast mailbox parsing

[Skip Montanaro]

    Kai> I try to find new messages in mailboxes. I use the mailbox and
    Kai> email module to parse the mailbox. Looping over all messages and
    Kai> looking for the Status header with msg.get('Status') is sadly very
    Kai> slow.

    Kai> Is there a faster way of parsing mailboxes in Python? Mutt does it
    Kai> so fast!

How big are your mailboxes?  Have you tried caching some of the information
so you don't have to continually go back and loop over everything?  I ran it
over my spam mailbox just now:

    % /usr/bin/time python statusproto.py newspam.clean.save 
    Total: 7002 - Old:    1 - New: 7001
           18.63 real        15.08 user         2.01 sys

I'm sure 375 msgs/sec isn't as fast as Mutt, but it seems fast enough for
me.

Skip