[Spambayes] new python error in sbfilter.py

[Fred Smith]

On Thu, Mar 09, 2017 at 07:23:38AM -0600, Skip Montanaro wrote:
>    Fred,
>    It looks like your training database is corrupt. At the very end of the
>    long traceback, the message indicates that the count of messages (ham
>    or spam) in which a particular word appears is greater than the number
>    of messages in that particular category. I think you should be able to
>    just retrain from scratch on your existing database.
>    Skip

Sigh.

That worked. for a little while. then it started doing it again.

I've recently started using these macros in mutt:

	macro index S "|sb_filter.py -s -f | procmail\&\nd"
	macro pager S "|sb_filter.py -s -f | procmail\&\nd"
	macro index H "|sb_filter.py -g -f | procmail\&\nd"
	macro pager H "|sb_filter.py -g -f | procmail\&\nd"

and in procmail there are these rules:

	:0 fw:hamlock
	| /usr/bin/sb_filter.py -f -d $HOME/.hammiedb

	# then filter out spam and unsure stuff....
	:0
	* ^X-Spambayes-Classification: spam
	$HOME/Mail/trained.spam

	:0
	* ^X-Spambayes-Classification: unsure
	$HOME/Mail/unsure

I don't see why those macros would cause such a problem, but it
has started only since I started using them (of course, I also blew
away the ancient hammie db and started over with a small corpus of
known ham and spam, at the same time).

Prior to that I would just save mis-filed mails in either trained.spam
or trained.ham and trust that the nightly retraining would do the right
thing.

any further ideas?

thanks in advance!

Fred

> 
>    On Mar 8, 2017 7:11 PM, "Fred Smith" <[1]fredex at fcshome.stoneham.ma.us>
>    wrote:
> 
>      Hi
>      All of a sudden this past week I'm getting this whenever a message
>      is
>      sent to sb_filter to be retrained:
>      File "/usr/bin/sb_filter.py", line 5, in <module>
>          pkg_resources.run_script('spambayes==1.1a6', 'sb_filter.py')
>        File "/usr/lib/python2.7/site-packages/pkg_resources.py", line
>      540, in run_script
>          self.require(requires)[0].run_script(script_name, ns)
>        File "/usr/lib/python2.7/site-packages/pkg_resources.py", line
>      1455, in run_script
>          execfile(script_filename, namespace, namespace)
>        File "/usr/lib/python2.7/site-packages/spambayes-1.1a6-py2.
>      7.egg/EGG-INFO/scripts/sb_filter.py", line 277, in <module>
>          main()
>        File "/usr/lib/python2.7/site-packages/spambayes-1.1a6-py2.
>      7.egg/EGG-INFO/scripts/sb_filter.py", line 268, in main
>          action(msg)
>        File "/usr/lib/python2.7/site-packages/spambayes-1.1a6-py2.
>      7.egg/EGG-INFO/scripts/sb_filter.py", line 186, in filter
>          return self.h.filter(msg)
>        File "/usr/lib/python2.7/site-packages/spambayes-1.1a6-py2.
>      7.egg/spambayes/hammie.py", line 149, in filter
>          debug, train)
>        File "/usr/lib/python2.7/site-packages/spambayes-1.1a6-py2.
>      7.egg/spambayes/hammie.py", line 104, in score_and_filter
>          prob, clues = self._scoremsg(msg, True)
>        File "/usr/lib/python2.7/site-packages/spambayes-1.1a6-py2.
>      7.egg/spambayes/hammie.py", line 33, in _scoremsg
>          return self.bayes.spamprob(tokenize(msg), evidence)
>        File "/usr/lib/python2.7/site-packages/spambayes-1.1a6-py2.
>      7.egg/spambayes/classifier.py", line 169, in chi2_spamprob
>          clues = self._getclues(wordstream)
>        File "/usr/lib/python2.7/site-packages/spambayes-1.1a6-py2.
>      7.egg/spambayes/classifier.py", line 472, in _getclues
>          tup = self._worddistanceget(word)
>        File "/usr/lib/python2.7/site-packages/spambayes-1.1a6-py2.
>      7.egg/spambayes/classifier.py", line 487, in _worddistanceget
>          prob = self.probability(record)
>        File "/usr/lib/python2.7/site-packages/spambayes-1.1a6-py2.
>      7.egg/spambayes/classifier.py", line 287, in probability
>          assert hamcount <= nham, "Token seen in more ham than ham
>      trained."
>      AssertionError: Token seen in more ham than ham trained.
>      It is possible I got a python update, but I wasn't paying attention,
>      so
>      I'm not at all sure.
>      I'm NOT a python guru, so I'd appreciate any guidance any of you can
>      provide.
>      thanks in advance!
>      Fred
>      --
>      ---- Fred Smith -- [2]fredex at fcshome.stoneham.ma.us
>      -----------------------------
>                          The Lord detests the way of the wicked
>                        but he loves those who pursue righteousness.
>      ----------------------------- Proverbs 15:9 (niv)
>      -----------------------------
>      _______________________________________________
>      [3]SpamBayes at python.org
>      [4]https://mail.python.org/mailman/listinfo/spambayes
>      Info/Unsubscribe: [5]http://mail.python.org/
>      mailman/listinfo/spambayes
>      Check the FAQ before asking: [6]http://spambayes.sf.net/faq.html
> 
> References
> 
>    1. mailto:fredex at fcshome.stoneham.ma.us
>    2. mailto:fredex at fcshome.stoneham.ma.us
>    3. mailto:SpamBayes at python.org
>    4. https://mail.python.org/mailman/listinfo/spambayesInfo/Unsubscribe
>    5. http://mail.python.org/mailman/listinfo/spambayes
>    6. http://spambayes.sf.net/faq.html

-- 
---- Fred Smith -- fredex at fcshome.stoneham.ma.us -----------------------------
                        The Lord is like a strong tower. 
             Those who do what is right can run to him for safety.
--------------------------- Proverbs 18:10 (niv) -----------------------------