Third result ... RE: [Spambayes] First result from Gary Robinson's ideas

[Anthony Baxter]

Let's try that again, this time with the current tokenizer.py

same settings as before.

false positive percentages
    0.223  0.334  lost   +49.78%
    0.278  0.278  tied
    0.167  0.167  tied
    0.278  0.278  tied
    0.334  0.389  lost   +16.47%

won   0 times
tied  3 times
lost  2 times

total unique fp went from 23 to 26 lost   +13.04%
mean fp % went from 0.255790439419 to 0.289173231033 lost   +13.05%

false negative percentages
    0.582  0.517  won    -11.17%
    0.388  0.388  tied
    0.581  0.516  won    -11.19%
    0.518  0.453  won    -12.55%
    0.712  0.712  tied

won   3 times
tied  2 times
lost  0 times

total unique fn went from 43 to 40 won     -6.98%
mean fn % went from 0.556199665243 to 0.517406493303 won     -6.97%

before:
Ham distribution for all runs:
* = 150 items
  0.00 8961 ************************************************************
  2.50    1 *
  5.00    0 
  7.50    2 *
 10.00    0 
 12.50    0 
 15.00    0 
 17.50    0 
 20.00    0 
 22.50    0 
 25.00    0 
 27.50    0 
 30.00    0 
 32.50    0 
 35.00    0 
 37.50    0 
 40.00    0 
 42.50    0 
 45.00    0 
 47.50    1 *
 50.00    1 *
 52.50    1 *
 55.00    0 
 57.50    0 
 60.00    0 
 62.50    0 
 65.00    0 
 67.50    0 
 70.00    0 
 72.50    0 
 75.00    0 
 77.50    0 
 80.00    0 
 82.50    0 
 85.00    1 *
 87.50    0 
 90.00    0 
 92.50    0 
 95.00    1 *
 97.50   22 *

Spam distribution for all runs:
* = 129 items
  0.00   35 *
  2.50    2 *
  5.00    0 
  7.50    2 *
 10.00    0 
 12.50    0 
 15.00    0 
 17.50    0 
 20.00    0 
 22.50    0 
 25.00    0 
 27.50    0 
 30.00    0 
 32.50    0 
 35.00    0 
 37.50    1 *
 40.00    0 
 42.50    0 
 45.00    0 
 47.50    0 
 50.00    0 
 52.50    0 
 55.00    0 
 57.50    1 *
 60.00    0 
 62.50    1 *
 65.00    0 
 67.50    1 *
 70.00    0 
 72.50    0 
 75.00    0 
 77.50    0 
 80.00    0 
 82.50    0 
 85.00    0 
 87.50    0 
 90.00    0 
 92.50    0 
 95.00    0 
 97.50 7688 ************************************************************


After:
Ham distribution for all runs:
* = 108 items
  0.00 6459 ************************************************************
  2.50  862 ********
  5.00  263 ***
  7.50   84 *
 10.00   32 *
 12.50  110 **
 15.00  379 ****
 17.50  241 ***
 20.00   80 *
 22.50   95 *
 25.00  116 **
 27.50   65 *
 30.00   48 *
 32.50   35 *
 35.00   23 *
 37.50   23 *
 40.00   16 *
 42.50   16 *
 45.00    9 *
 47.50    9 *
 50.00    7 *
 52.50    4 *
 55.00    7 *
 57.50    2 *
 60.00    1 *
 62.50    1 *
 65.00    0 
 67.50    0 
 70.00    0 
 72.50    1 *
 75.00    0 
 77.50    0 
 80.00    0 
 82.50    0 
 85.00    0 
 87.50    0 
 90.00    0 
 92.50    0 
 95.00    0 
 97.50    3 *

Spam distribution for all runs:
* = 123 items
  0.00    4 *
  2.50    0 
  5.00    0 
  7.50    0 
 10.00    0 
 12.50    0 
 15.00    1 *
 17.50    1 *
 20.00    0 
 22.50    1 *
 25.00    2 *
 27.50    2 *
 30.00    0 
 32.50    4 *
 35.00    3 *
 37.50    1 *
 40.00    3 *
 42.50    3 *
 45.00    5 *
 47.50   10 *
 50.00    7 *
 52.50   15 *
 55.00   15 *
 57.50   15 *
 60.00   16 *
 62.50   25 *
 65.00   27 *
 67.50   19 *
 70.00   20 *
 72.50   19 *
 75.00   21 *
 77.50   22 *
 80.00   44 *
 82.50   42 *
 85.00   26 *
 87.50    4 *
 90.00    0 
 92.50    1 *
 95.00   22 *
 97.50 7331 ************************************************************