[Spambayes-checkins] spambayes Options.py,1.61,1.62tokenizer.py,1.52,1.53

Tim Peters tim.one@comcast.net
Mon Oct 28 07:20:26 2002


[Anthony Baxter]
> Modified Files:
> 	Options.py tokenizer.py
> Log Message:
> Added skip_max_word_size as an option, to specify how long a word
> has to be before it's skipped. I find that boosting from 12 (the default)
> to 20 makes a significant improvement in the number of 'unsure' messages.
> see my post to the list for more.

This patch is confused, as it left the hardcoded 12 in tokenize_body()
alone.  You should also update the "How big should 'a word' be?" comment
block if you've found that boosting this helped.