From noreply at sourceforge.net  Tue Oct  2 00:46:39 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 01 Oct 2007 15:46:39 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1600821 ] Classifier
	UnicodeDecodeError on wrong transfer encoding
Message-ID: <E1IcU2B-0008C2-5T@sc8-sf-web24.sourceforge.net>

Bugs item #1600821, was opened at 2006-11-22 00:59
Message generated for change (Comment added) made by jcea
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1600821&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: imapfilter
Group: 1.0.1
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ivan Vilata i Balaguer (ivilata)
Assigned to: Tony Meyer (anadelonbrin)
Summary: Classifier UnicodeDecodeError on wrong transfer encoding

Initial Comment:
Running ``sb_imapfilter.py`` 1.0.1 seems to raise the following ``UnicodeDecodeError`` when it comes across a mail with 7-bit content transfer encoding with 8-bit characters in it while classifying::

    Traceback (most recent call last):
    File "/usr/bin/sb_imapfilter.py", line 924, in ?
      run()
    File "/usr/bin/sb_imapfilter.py", line 914, in run
      imap_filter.Filter()
    File "/usr/bin/sb_imapfilter.py", line 785, in Filter
      self.unsure_folder)
    File "/usr/bin/sb_imapfilter.py", line 703, in Filter
      evidence=True)
    File "/usr/lib/python2.4/site-packages/spambayes/classifier.py", line 190, in chi2_spamprob
      clues = self._getclues(wordstream)
    File "/usr/lib/python2.4/site-packages/spambayes/classifier.py", line 496, in _getclues
      clues.sort()
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: ordinal not in range(128)

I'm attaching the mail which caused this.  I know it is not properly-formatted, but it is a legitimate mail produced by a popular MUA (Thunderbird 1.5).  Spam surely is worsely formatted

Someone talked about the same problem in the list: http://www.mail-archive.com/spambayes at python.org/msg04543.html

----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-10-02 00:46

Message:
Logged In: YES 
user_id=97460
Originator: NO

Three examples sent to skip at pobox.com.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-26 04:48

Message:
Logged In: YES 
user_id=44345
Originator: NO

jcea,

Do you have an email message I can work with?  If so, zip it and send it
to me as an attachment (skip at pobox.com).

Thx,

Skip


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-26 04:02

Message:
Logged In: YES 
user_id=97460
Originator: NO

My version is 1.0.4 and the traceback is:

"""
Traceback (most recent call last):
  File "/usr/local/lib/python2.5/site-packages/Milter/__init__.py", line
203, in <lambda>
    milter.set_eom_callback(lambda ctx: ctx.getpriv().eom())
  File "antispam.py", line 513, in eom
    prob=hammiedb.score(msg)
  File "/usr/local/lib/python2.5/site-packages/spambayes/hammie.py", line
62, in score
    return self._scoremsg(msg, evidence)
  File "/usr/local/lib/python2.5/site-packages/spambayes/hammie.py", line
38, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File "/usr/local/lib/python2.5/site-packages/spambayes/classifier.py",
line 190, in chi2_spamprob
    clues = self._getclues(wordstream)
  File "/usr/local/lib/python2.5/site-packages/spambayes/classifier.py",
line 496, in _getclues
    clues.sort()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfa in position 0:
ordinal not in range(128)
"""


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-17 17:37

Message:
Logged In: YES 
user_id=97460
Originator: NO

My version is 1.0.4.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-07 04:11

Message:
Logged In: YES 
user_id=44345
Originator: NO

I ran the submitted email through the current sb_filter.py in Subversion
(probably the same classifier as in 1.1a4).  It worked for me.  While I
don't use the IMAP filter, any of the SpamBayes applications should use the
same classifier code.  I'm not sure this is a problem in the current code. 
What version of SpamBayes are you using?


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-05 19:23

Message:
Logged In: YES 
user_id=44345
Originator: NO

Do you have a traceback?  What version of SpamBayes are you using?


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-05 16:59

Message:
Logged In: YES 
user_id=97460
Originator: NO

I'm seeing a lot (>1 per hour in my system) of current spam crashing
spambayes because they are marked as "ascii" but body is 8-bit actually.

Since my milter spam filter crashes and sendmail disables the milter
filtering for 50 seconds because the failure (my configuration, and I
wouldn't like to touch it), a lot of spam is getting thru. About 30-100
spams, everytime this bug hits.

Please, increase the priority of this bug a bit... It is hitting. Hard.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1600821&group_id=61702

From noreply at sourceforge.net  Tue Oct  2 02:04:16 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 01 Oct 2007 17:04:16 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1600821 ] Classifier
	UnicodeDecodeError on wrong transfer encoding
Message-ID: <E1IcVFI-00088R-Jp@sc8-sf-web24.sourceforge.net>

Bugs item #1600821, was opened at 2006-11-21 17:59
Message generated for change (Settings changed) made by montanaro
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1600821&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: imapfilter
Group: 1.0.1
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ivan Vilata i Balaguer (ivilata)
>Assigned to: Skip Montanaro (montanaro)
Summary: Classifier UnicodeDecodeError on wrong transfer encoding

Initial Comment:
Running ``sb_imapfilter.py`` 1.0.1 seems to raise the following ``UnicodeDecodeError`` when it comes across a mail with 7-bit content transfer encoding with 8-bit characters in it while classifying::

    Traceback (most recent call last):
    File "/usr/bin/sb_imapfilter.py", line 924, in ?
      run()
    File "/usr/bin/sb_imapfilter.py", line 914, in run
      imap_filter.Filter()
    File "/usr/bin/sb_imapfilter.py", line 785, in Filter
      self.unsure_folder)
    File "/usr/bin/sb_imapfilter.py", line 703, in Filter
      evidence=True)
    File "/usr/lib/python2.4/site-packages/spambayes/classifier.py", line 190, in chi2_spamprob
      clues = self._getclues(wordstream)
    File "/usr/lib/python2.4/site-packages/spambayes/classifier.py", line 496, in _getclues
      clues.sort()
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: ordinal not in range(128)

I'm attaching the mail which caused this.  I know it is not properly-formatted, but it is a legitimate mail produced by a popular MUA (Thunderbird 1.5).  Spam surely is worsely formatted

Someone talked about the same problem in the list: http://www.mail-archive.com/spambayes at python.org/msg04543.html

----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-10-01 17:46

Message:
Logged In: YES 
user_id=97460
Originator: NO

Three examples sent to skip at pobox.com.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-25 21:48

Message:
Logged In: YES 
user_id=44345
Originator: NO

jcea,

Do you have an email message I can work with?  If so, zip it and send it
to me as an attachment (skip at pobox.com).

Thx,

Skip


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-25 21:02

Message:
Logged In: YES 
user_id=97460
Originator: NO

My version is 1.0.4 and the traceback is:

"""
Traceback (most recent call last):
  File "/usr/local/lib/python2.5/site-packages/Milter/__init__.py", line
203, in <lambda>
    milter.set_eom_callback(lambda ctx: ctx.getpriv().eom())
  File "antispam.py", line 513, in eom
    prob=hammiedb.score(msg)
  File "/usr/local/lib/python2.5/site-packages/spambayes/hammie.py", line
62, in score
    return self._scoremsg(msg, evidence)
  File "/usr/local/lib/python2.5/site-packages/spambayes/hammie.py", line
38, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File "/usr/local/lib/python2.5/site-packages/spambayes/classifier.py",
line 190, in chi2_spamprob
    clues = self._getclues(wordstream)
  File "/usr/local/lib/python2.5/site-packages/spambayes/classifier.py",
line 496, in _getclues
    clues.sort()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfa in position 0:
ordinal not in range(128)
"""


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-17 10:37

Message:
Logged In: YES 
user_id=97460
Originator: NO

My version is 1.0.4.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-06 21:11

Message:
Logged In: YES 
user_id=44345
Originator: NO

I ran the submitted email through the current sb_filter.py in Subversion
(probably the same classifier as in 1.1a4).  It worked for me.  While I
don't use the IMAP filter, any of the SpamBayes applications should use the
same classifier code.  I'm not sure this is a problem in the current code. 
What version of SpamBayes are you using?


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-05 12:23

Message:
Logged In: YES 
user_id=44345
Originator: NO

Do you have a traceback?  What version of SpamBayes are you using?


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-05 09:59

Message:
Logged In: YES 
user_id=97460
Originator: NO

I'm seeing a lot (>1 per hour in my system) of current spam crashing
spambayes because they are marked as "ascii" but body is 8-bit actually.

Since my milter spam filter crashes and sendmail disables the milter
filtering for 50 seconds because the failure (my configuration, and I
wouldn't like to touch it), a lot of spam is getting thru. About 30-100
spams, everytime this bug hits.

Please, increase the priority of this bug a bit... It is hitting. Hard.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1600821&group_id=61702

From noreply at sourceforge.net  Tue Oct  2 02:06:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 01 Oct 2007 17:06:04 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1600821 ] Classifier
	UnicodeDecodeError on wrong transfer encoding
Message-ID: <E1IcVH2-0008LO-OU@sc8-sf-web24.sourceforge.net>

Bugs item #1600821, was opened at 2006-11-21 17:59
Message generated for change (Comment added) made by montanaro
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1600821&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: imapfilter
Group: 1.0.1
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ivan Vilata i Balaguer (ivilata)
Assigned to: Skip Montanaro (montanaro)
Summary: Classifier UnicodeDecodeError on wrong transfer encoding

Initial Comment:
Running ``sb_imapfilter.py`` 1.0.1 seems to raise the following ``UnicodeDecodeError`` when it comes across a mail with 7-bit content transfer encoding with 8-bit characters in it while classifying::

    Traceback (most recent call last):
    File "/usr/bin/sb_imapfilter.py", line 924, in ?
      run()
    File "/usr/bin/sb_imapfilter.py", line 914, in run
      imap_filter.Filter()
    File "/usr/bin/sb_imapfilter.py", line 785, in Filter
      self.unsure_folder)
    File "/usr/bin/sb_imapfilter.py", line 703, in Filter
      evidence=True)
    File "/usr/lib/python2.4/site-packages/spambayes/classifier.py", line 190, in chi2_spamprob
      clues = self._getclues(wordstream)
    File "/usr/lib/python2.4/site-packages/spambayes/classifier.py", line 496, in _getclues
      clues.sort()
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: ordinal not in range(128)

I'm attaching the mail which caused this.  I know it is not properly-formatted, but it is a legitimate mail produced by a popular MUA (Thunderbird 1.5).  Spam surely is worsely formatted

Someone talked about the same problem in the list: http://www.mail-archive.com/spambayes at python.org/msg04543.html

----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2007-10-01 19:06

Message:
Logged In: YES 
user_id=44345
Originator: NO

File Added: mailbox

----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-10-01 17:46

Message:
Logged In: YES 
user_id=97460
Originator: NO

Three examples sent to skip at pobox.com.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-25 21:48

Message:
Logged In: YES 
user_id=44345
Originator: NO

jcea,

Do you have an email message I can work with?  If so, zip it and send it
to me as an attachment (skip at pobox.com).

Thx,

Skip


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-25 21:02

Message:
Logged In: YES 
user_id=97460
Originator: NO

My version is 1.0.4 and the traceback is:

"""
Traceback (most recent call last):
  File "/usr/local/lib/python2.5/site-packages/Milter/__init__.py", line
203, in <lambda>
    milter.set_eom_callback(lambda ctx: ctx.getpriv().eom())
  File "antispam.py", line 513, in eom
    prob=hammiedb.score(msg)
  File "/usr/local/lib/python2.5/site-packages/spambayes/hammie.py", line
62, in score
    return self._scoremsg(msg, evidence)
  File "/usr/local/lib/python2.5/site-packages/spambayes/hammie.py", line
38, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File "/usr/local/lib/python2.5/site-packages/spambayes/classifier.py",
line 190, in chi2_spamprob
    clues = self._getclues(wordstream)
  File "/usr/local/lib/python2.5/site-packages/spambayes/classifier.py",
line 496, in _getclues
    clues.sort()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfa in position 0:
ordinal not in range(128)
"""


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-17 10:37

Message:
Logged In: YES 
user_id=97460
Originator: NO

My version is 1.0.4.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-06 21:11

Message:
Logged In: YES 
user_id=44345
Originator: NO

I ran the submitted email through the current sb_filter.py in Subversion
(probably the same classifier as in 1.1a4).  It worked for me.  While I
don't use the IMAP filter, any of the SpamBayes applications should use the
same classifier code.  I'm not sure this is a problem in the current code. 
What version of SpamBayes are you using?


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-05 12:23

Message:
Logged In: YES 
user_id=44345
Originator: NO

Do you have a traceback?  What version of SpamBayes are you using?


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-05 09:59

Message:
Logged In: YES 
user_id=97460
Originator: NO

I'm seeing a lot (>1 per hour in my system) of current spam crashing
spambayes because they are marked as "ascii" but body is 8-bit actually.

Since my milter spam filter crashes and sendmail disables the milter
filtering for 50 seconds because the failure (my configuration, and I
wouldn't like to touch it), a lot of spam is getting thru. About 30-100
spams, everytime this bug hits.

Please, increase the priority of this bug a bit... It is hitting. Hard.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1600821&group_id=61702

From noreply at sourceforge.net  Tue Oct  2 04:16:29 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 01 Oct 2007 19:16:29 -0700
Subject: [spambayes-bugs] [ spambayes-Feature Requests-1806056 ] Option to
	move ham
Message-ID: <E1IcXJF-0008UE-Qv@sc8-sf-web21.sourceforge.net>

Feature Requests item #1806056, was opened at 2007-10-01 19:16
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1806056&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Outlook
Group: Next Release (example)
Status: Open
Priority: 5
Private: No
Submitted By: onsitehelp (onsitehelp)
Assigned to: Mark Hammond (mhammond)
Summary: Option to move ham

Initial Comment:
Hi,

I use an IMAP IDLE based PDA phone email client, and find I keep getting new mail notifications, even though the message gets moved out of the inbox shortly after it arrives from SpamBayes doing its work.

Since I can set my IMAP IDLE to watch any folder, it would sure be nice to be able to specify that ham should be moved to a specified folder, so my phone doesn't keep notifying me of a new message whenever spam shows up.

Some ideas: Be able to specify one ham folder that receives all identified ham messages. Or, do this plus allow specifying a ham folder for each identified folder that is being watched. Or, it could be a simple checkbox option to "Enable Ham subfolder", which creates a "ham" subfolder under the folder being watched, so each folder being watched would automatically get its own automatically created ham-dedicated.

Eric Taneda
Onsitehelp.com


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1806056&group_id=61702

From noreply at sourceforge.net  Tue Oct  2 04:20:20 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 01 Oct 2007 19:20:20 -0700
Subject: [spambayes-bugs] [ spambayes-Feature Requests-1806056 ] Option to
	move ham
Message-ID: <E1IcXMy-00020Y-Bf@sc8-sf-web22.sourceforge.net>

Feature Requests item #1806056, was opened at 2007-10-01 19:16
Message generated for change (Comment added) made by onsitehelp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1806056&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Outlook
Group: Next Release (example)
Status: Open
Priority: 5
Private: No
Submitted By: onsitehelp (onsitehelp)
Assigned to: Mark Hammond (mhammond)
Summary: Option to move ham

Initial Comment:
Hi,

I use an IMAP IDLE based PDA phone email client, and find I keep getting new mail notifications, even though the message gets moved out of the inbox shortly after it arrives from SpamBayes doing its work.

Since I can set my IMAP IDLE to watch any folder, it would sure be nice to be able to specify that ham should be moved to a specified folder, so my phone doesn't keep notifying me of a new message whenever spam shows up.

Some ideas: Be able to specify one ham folder that receives all identified ham messages. Or, do this plus allow specifying a ham folder for each identified folder that is being watched. Or, it could be a simple checkbox option to "Enable Ham subfolder", which creates a "ham" subfolder under the folder being watched, so each folder being watched would automatically get its own automatically created ham-dedicated.

Eric Taneda
Onsitehelp.com


----------------------------------------------------------------------

>Comment By: onsitehelp (onsitehelp)
Date: 2007-10-01 19:20

Message:
Logged In: YES 
user_id=1903421
Originator: YES

The last sentence should have ended with ..."so each folder being watched
would automatically get its own automatically created ham-dedicated
folder."


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1806056&group_id=61702

From noreply at sourceforge.net  Tue Oct  2 05:44:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 01 Oct 2007 20:44:04 -0700
Subject: [spambayes-bugs] [ spambayes-Feature Requests-1806056 ] Option to
	move ham
Message-ID: <E1IcYg0-0005Pt-7w@sc8-sf-web23.sourceforge.net>

Feature Requests item #1806056, was opened at 2007-10-02 12:16
Message generated for change (Comment added) made by mhammond
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1806056&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Outlook
Group: Next Release (example)
>Status: Pending
Priority: 5
Private: No
Submitted By: onsitehelp (onsitehelp)
Assigned to: Mark Hammond (mhammond)
Summary: Option to move ham

Initial Comment:
Hi,

I use an IMAP IDLE based PDA phone email client, and find I keep getting new mail notifications, even though the message gets moved out of the inbox shortly after it arrives from SpamBayes doing its work.

Since I can set my IMAP IDLE to watch any folder, it would sure be nice to be able to specify that ham should be moved to a specified folder, so my phone doesn't keep notifying me of a new message whenever spam shows up.

Some ideas: Be able to specify one ham folder that receives all identified ham messages. Or, do this plus allow specifying a ham folder for each identified folder that is being watched. Or, it could be a simple checkbox option to "Enable Ham subfolder", which creates a "ham" subfolder under the folder being watched, so each folder being watched would automatically get its own automatically created ham-dedicated.

Eric Taneda
Onsitehelp.com


----------------------------------------------------------------------

>Comment By: Mark Hammond (mhammond)
Date: 2007-10-02 13:44

Message:
Logged In: YES 
user_id=14198
Originator: NO

The latest release of the Outlook plugin already has this feature - you
can specify a folder where all Ham is to be moved.  Please grab 1.1a4 from
sourceforge, and please close this bug if that satisfies your request.

----------------------------------------------------------------------

Comment By: onsitehelp (onsitehelp)
Date: 2007-10-02 12:20

Message:
Logged In: YES 
user_id=1903421
Originator: YES

The last sentence should have ended with ..."so each folder being watched
would automatically get its own automatically created ham-dedicated
folder."


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1806056&group_id=61702

From noreply at sourceforge.net  Tue Oct  2 12:55:51 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 02 Oct 2007 03:55:51 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1600821 ] Classifier
	UnicodeDecodeError on wrong transfer encoding
Message-ID: <E1IcfPr-0005qJ-H3@sc8-sf-web22.sourceforge.net>

Bugs item #1600821, was opened at 2006-11-21 17:59
Message generated for change (Comment added) made by montanaro
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1600821&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: imapfilter
Group: 1.0.1
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ivan Vilata i Balaguer (ivilata)
Assigned to: Skip Montanaro (montanaro)
Summary: Classifier UnicodeDecodeError on wrong transfer encoding

Initial Comment:
Running ``sb_imapfilter.py`` 1.0.1 seems to raise the following ``UnicodeDecodeError`` when it comes across a mail with 7-bit content transfer encoding with 8-bit characters in it while classifying::

    Traceback (most recent call last):
    File "/usr/bin/sb_imapfilter.py", line 924, in ?
      run()
    File "/usr/bin/sb_imapfilter.py", line 914, in run
      imap_filter.Filter()
    File "/usr/bin/sb_imapfilter.py", line 785, in Filter
      self.unsure_folder)
    File "/usr/bin/sb_imapfilter.py", line 703, in Filter
      evidence=True)
    File "/usr/lib/python2.4/site-packages/spambayes/classifier.py", line 190, in chi2_spamprob
      clues = self._getclues(wordstream)
    File "/usr/lib/python2.4/site-packages/spambayes/classifier.py", line 496, in _getclues
      clues.sort()
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1: ordinal not in range(128)

I'm attaching the mail which caused this.  I know it is not properly-formatted, but it is a legitimate mail produced by a popular MUA (Thunderbird 1.5).  Spam surely is worsely formatted

Someone talked about the same problem in the list: http://www.mail-archive.com/spambayes at python.org/msg04543.html

----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2007-10-02 05:55

Message:
Logged In: YES 
user_id=44345
Originator: NO

None of these make the current version of sb_filter.py barf.
I wonder if there's something peculiar about the way the
mail is transmitted via IMAP?  (Just a wild guess.)


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-10-01 19:06

Message:
Logged In: YES 
user_id=44345
Originator: NO

File Added: mailbox

----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-10-01 17:46

Message:
Logged In: YES 
user_id=97460
Originator: NO

Three examples sent to skip at pobox.com.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-25 21:48

Message:
Logged In: YES 
user_id=44345
Originator: NO

jcea,

Do you have an email message I can work with?  If so, zip it and send it
to me as an attachment (skip at pobox.com).

Thx,

Skip


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-25 21:02

Message:
Logged In: YES 
user_id=97460
Originator: NO

My version is 1.0.4 and the traceback is:

"""
Traceback (most recent call last):
  File "/usr/local/lib/python2.5/site-packages/Milter/__init__.py", line
203, in <lambda>
    milter.set_eom_callback(lambda ctx: ctx.getpriv().eom())
  File "antispam.py", line 513, in eom
    prob=hammiedb.score(msg)
  File "/usr/local/lib/python2.5/site-packages/spambayes/hammie.py", line
62, in score
    return self._scoremsg(msg, evidence)
  File "/usr/local/lib/python2.5/site-packages/spambayes/hammie.py", line
38, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File "/usr/local/lib/python2.5/site-packages/spambayes/classifier.py",
line 190, in chi2_spamprob
    clues = self._getclues(wordstream)
  File "/usr/local/lib/python2.5/site-packages/spambayes/classifier.py",
line 496, in _getclues
    clues.sort()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xfa in position 0:
ordinal not in range(128)
"""


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-17 10:37

Message:
Logged In: YES 
user_id=97460
Originator: NO

My version is 1.0.4.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-06 21:11

Message:
Logged In: YES 
user_id=44345
Originator: NO

I ran the submitted email through the current sb_filter.py in Subversion
(probably the same classifier as in 1.1a4).  It worked for me.  While I
don't use the IMAP filter, any of the SpamBayes applications should use the
same classifier code.  I'm not sure this is a problem in the current code. 
What version of SpamBayes are you using?


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-09-05 12:23

Message:
Logged In: YES 
user_id=44345
Originator: NO

Do you have a traceback?  What version of SpamBayes are you using?


----------------------------------------------------------------------

Comment By: Jes?s Cea Avi?n (jcea)
Date: 2007-09-05 09:59

Message:
Logged In: YES 
user_id=97460
Originator: NO

I'm seeing a lot (>1 per hour in my system) of current spam crashing
spambayes because they are marked as "ascii" but body is 8-bit actually.

Since my milter spam filter crashes and sendmail disables the milter
filtering for 50 seconds because the failure (my configuration, and I
wouldn't like to touch it), a lot of spam is getting thru. About 30-100
spams, everytime this bug hits.

Please, increase the priority of this bug a bit... It is hitting. Hard.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1600821&group_id=61702

From noreply at sourceforge.net  Tue Oct  2 16:04:22 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 02 Oct 2007 07:04:22 -0700
Subject: [spambayes-bugs] [ spambayes-Feature Requests-1806056 ] Option to
	move ham
Message-ID: <E1IciMI-0006GH-3K@sc8-sf-web24.sourceforge.net>

Feature Requests item #1806056, was opened at 2007-10-01 19:16
Message generated for change (Settings changed) made by onsitehelp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1806056&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Outlook
Group: Next Release (example)
>Status: Closed
Priority: 5
Private: No
Submitted By: onsitehelp (onsitehelp)
Assigned to: Mark Hammond (mhammond)
Summary: Option to move ham

Initial Comment:
Hi,

I use an IMAP IDLE based PDA phone email client, and find I keep getting new mail notifications, even though the message gets moved out of the inbox shortly after it arrives from SpamBayes doing its work.

Since I can set my IMAP IDLE to watch any folder, it would sure be nice to be able to specify that ham should be moved to a specified folder, so my phone doesn't keep notifying me of a new message whenever spam shows up.

Some ideas: Be able to specify one ham folder that receives all identified ham messages. Or, do this plus allow specifying a ham folder for each identified folder that is being watched. Or, it could be a simple checkbox option to "Enable Ham subfolder", which creates a "ham" subfolder under the folder being watched, so each folder being watched would automatically get its own automatically created ham-dedicated.

Eric Taneda
Onsitehelp.com


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2007-10-01 20:44

Message:
Logged In: YES 
user_id=14198
Originator: NO

The latest release of the Outlook plugin already has this feature - you
can specify a folder where all Ham is to be moved.  Please grab 1.1a4 from
sourceforge, and please close this bug if that satisfies your request.

----------------------------------------------------------------------

Comment By: onsitehelp (onsitehelp)
Date: 2007-10-01 19:20

Message:
Logged In: YES 
user_id=1903421
Originator: YES

The last sentence should have ended with ..."so each folder being watched
would automatically get its own automatically created ham-dedicated
folder."


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1806056&group_id=61702

From noreply at sourceforge.net  Thu Oct 11 21:06:23 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 11 Oct 2007 12:06:23 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1811774 ] Outlook plugin doesn't
	load
Message-ID: <E1Ig3MV-0002sJ-7h@sc8-sf-web24.sourceforge.net>

Bugs item #1811774, was opened at 2007-10-11 12:06
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1811774&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Outlook
Group: 1.0.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: joefidler (joefidler)
Assigned to: Nobody/Anonymous (nobody)
Summary: Outlook plugin doesn't load

Initial Comment:
Installed Outlook plugin with Outlook 2003 from the windows binary installer. Outlook is unable to load plugin and disables it upon startup. No error is reported in SpamBayes log (see below) and Outlook only reports that a plug-in was unable to load.

As a work around I am using the proxy service and it is working fine. Thanks for your help and a your efforts in general on SpamBayes

versions:
SpamBayes 1.0.4, then tried 1.1a4 with same results
Outlook/Office 2003 - SP3
Windows XP SP2
Hardware - AMD Turion 64 with 1 Gig RAM

SpamBayes Log :
Registered: SpamBayes.OutlookAddin 
Registration complete.
Registration (in HKEY_LOCAL_MACHINE) complete.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1811774&group_id=61702

From noreply at sourceforge.net  Fri Oct 12 08:15:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 11 Oct 2007 23:15:52 -0700
Subject: [spambayes-bugs] [ spambayes-Feature Requests-1643132 ] Icon color
	change when mail is received
Message-ID: <E1IgDoO-00030I-Cr@sc8-sf-web24.sourceforge.net>

Feature Requests item #1643132, was opened at 2007-01-23 20:41
Message generated for change (Comment added) made by hillbilly1980
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1643132&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Interface Improvements (example)
Group: Next Release (example)
Status: Open
Priority: 5
Private: No
Submitted By: Juan T. Llibre (juanllibre)
Assigned to: Nobody/Anonymous (nobody)
Summary: Icon color change when mail is received

Initial Comment:
Having the tray icon change to a different color, 
as a way to signal that mail is available for
classification/training, would be an easy way
for users to know when to check/start training.

----------------------------------------------------------------------

Comment By: Keith Page (hillbilly1980)
Date: 2007-10-12 00:15

Message:
Logged In: YES 
user_id=833282
Originator: NO

How about a new icon so i don't look unprofessional when i roll out
something great like this in a small but upscale corporate environment.
"The happy face means good the angry face means spam, isn't it cute... yes
it does work.. yes it's very advanced technology... ooh look it makes
shotgun sound when it moves junk mail. Booom boom take that spam."

I'm mean i know.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1643132&group_id=61702

From noreply at sourceforge.net  Sat Oct 13 16:09:28 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 13 Oct 2007 07:09:28 -0700
Subject: [spambayes-bugs] [ spambayes-Feature Requests-1643132 ] Icon color
	change when mail is received
Message-ID: <E1IghgG-0006f6-V9@sc8-sf-web24.sourceforge.net>

Feature Requests item #1643132, was opened at 2007-01-23 23:41
Message generated for change (Comment added) made by juanllibre
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1643132&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Interface Improvements (example)
Group: Next Release (example)
Status: Open
Priority: 5
Private: No
Submitted By: Juan T. Llibre (juanllibre)
Assigned to: Nobody/Anonymous (nobody)
Summary: Icon color change when mail is received

Initial Comment:
Having the tray icon change to a different color, 
as a way to signal that mail is available for
classification/training, would be an easy way
for users to know when to check/start training.

----------------------------------------------------------------------

>Comment By: Juan T. Llibre (juanllibre)
Date: 2007-10-13 10:09

Message:
Logged In: YES 
user_id=1700060
Originator: YES

re:
> I'm mean i know.

Nah, you're just childish.



----------------------------------------------------------------------

Comment By: Keith Page (hillbilly1980)
Date: 2007-10-12 02:15

Message:
Logged In: YES 
user_id=833282
Originator: NO

How about a new icon so i don't look unprofessional when i roll out
something great like this in a small but upscale corporate environment.
"The happy face means good the angry face means spam, isn't it cute... yes
it does work.. yes it's very advanced technology... ooh look it makes
shotgun sound when it moves junk mail. Booom boom take that spam."

I'm mean i know.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498106&aid=1643132&group_id=61702

From noreply at sourceforge.net  Fri Oct 19 06:36:37 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 18 Oct 2007 21:36:37 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-922063 ] Intermittent
	sb_filter.py failure with URL pickle
Message-ID: <E1IijbB-0003Sa-Gr@sc8-sf-web24.sourceforge.net>

Bugs item #922063, was opened at 2004-03-23 17:10
Message generated for change (Comment added) made by david_abrahams
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=922063&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: hammie
Group: Source code - CVS
Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Leonid (leobru)
Assigned to: Tony Meyer (anadelonbrin)
Summary: Intermittent sb_filter.py failure with URL pickle

Initial Comment:
Here are the relevant .spambayesrc lines:

[Tokenizer]
x-fancy_url_recognition=True
x-pick_apart_urls=True

[URLRetriever]
x-slurp_urls=True

Here is the stack trace:

  File
"/usr/home/leob/spambayes-1.0a9/scripts/sb_filter.py",
line 239, in ?
    main()
  File
"/usr/home/leob/spambayes-1.0a9/scripts/sb_filter.py",
line 231, in main
    action(msg)
  File
"/usr/home/leob/spambayes-1.0a9/scripts/sb_filter.py",
line 163, in filter
    return h.filter(msg)
  File
"/usr/home/leob/opt/lib/python2.2/site-packages/spambayes/hammie.py",
line 109, in filter
    prob, clues = self._scoremsg(msg, True)
  File
"/usr/home/leob/opt/lib/python2.2/site-packages/spambayes/hammie.py",
line 38, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File
"/usr/home/leob/opt/lib/python2.2/site-packages/spambayes/classifier.py",
line 246, in slurpi
ng_spamprob
    slurp_tokens = list(self._generate_slurp())
  File
"/usr/home/leob/opt/lib/python2.2/site-packages/spambayes/classifier.py",
line 550, in _gener
ate_slurp
    self.setup()
  File
"/usr/home/leob/opt/lib/python2.2/site-packages/spambayes/classifier.py",
line 609, in setup
    self.bad_urls = pickle.load(b_file)
  File "/usr/home/leob/opt/lib/python2.2/pickle.py",
line 982, in load
    return Unpickler(file).load()
  File "/usr/home/leob/opt/lib/python2.2/pickle.py",
line 597, in load
    dispatch[key](self)
  File "/usr/home/leob/opt/lib/python2.2/pickle.py",
line 667, in load_string
    raise ValueError, "insecure string pickle"


----------------------------------------------------------------------

Comment By: David Abrahams (david_abrahams)
Date: 2007-10-18 23:36

Message:
Logged In: YES 
user_id=52572
Originator: NO

I'm seeing the same problem in ImageStripper.py now:

saving 720 items to /home/dave/spambayes/imagecache.pck
Traceback (most recent call last):
  File "/usr/local/bin/sb_filter.py", line 290, in ?
    main()
  File "/usr/local/bin/sb_filter.py", line 281, in main
    action(msg)
  File "/usr/local/bin/sb_filter.py", line 199, in filter
    return self.h.filter(msg)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line
156, in filter
    debug, train)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line
110, in score_and_filter
    prob, clues = self._scoremsg(msg, True)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line
39, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File "/usr/local/lib/python2.4/site-packages/spambayes/classifier.py",
line 196, in chi2_spampro
b
    clues = self._getclues(wordstream)
  File "/usr/local/lib/python2.4/site-packages/spambayes/classifier.py",
line 498, in _getclues
    for word in Set(wordstream):
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py",
line 1281, in tokenize
    for tok in self.tokenize_body(msg):
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py",
line 1640, in tokenize_bod
y
    from spambayes.ImageStripper import crack_images
  File
"/usr/local/lib/python2.4/site-packages/spambayes/ImageStripper.py", line
391, in ?
    crack_images = ImageStripper(_cachefile).analyze
  File
"/usr/local/lib/python2.4/site-packages/spambayes/ImageStripper.py", line
305, in __init__
    self.cache = pickle.load(open(self.cachefile))
ValueError: insecure string pickle


----------------------------------------------------------------------

Comment By: Tony Meyer (anadelonbrin)
Date: 2004-11-02 20:11

Message:
Logged In: YES 
user_id=552329

For the sake of resolving this, I've changed the code anyway:

 1. If an error occurs loading the pickle, then a new one is
used - at least the classifier will keep going, and this
shouldn't hurt much (it's only a cache).

 2. Saving saves to a temp file first, and then replaces the
old one.  This should be completely (*nix) or reasonably
(win32) robust.

----------------------------------------------------------------------

Comment By: Tony Meyer (anadelonbrin)
Date: 2004-08-03 00:59

Message:
Logged In: YES 
user_id=552329

I'm guessing that something went wrong writing the pickle. 
(I get an EOFError trying to open the attached pickle).  The
slurping code really ought to do what the other code does
and save a copy and then replace the original once the save
succeeds.

I'm reluctant to do this at the moment, though, since it
seems fairly likely that the slurping code will vanish given
that it's only experimental and no-one's spoken up saying
that it does them any good.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=922063&group_id=61702

From noreply at sourceforge.net  Fri Oct 19 06:38:44 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 18 Oct 2007 21:38:44 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1816240 ] ImageStripper pickle
	corruption
Message-ID: <E1IijdE-0003fq-0b@sc8-sf-web24.sourceforge.net>

Bugs item #1816240, was opened at 2007-10-18 23:38
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1816240&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: hammie
Group: Source code - CVS
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: David Abrahams (david_abrahams)
Assigned to: Nobody/Anonymous (nobody)
Summary: ImageStripper pickle corruption

Initial Comment:
saving 720 items to /home/dave/spambayes/imagecache.pck
Traceback (most recent call last):
  File "/usr/local/bin/sb_filter.py", line 290, in ?
    main()
  File "/usr/local/bin/sb_filter.py", line 281, in main
    action(msg)
  File "/usr/local/bin/sb_filter.py", line 199, in filter
    return self.h.filter(msg)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 156, in filter
    debug, train)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 110, in score_and_filter
    prob, clues = self._scoremsg(msg, True)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 39, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File "/usr/local/lib/python2.4/site-packages/spambayes/classifier.py", line 196, in chi2_spampro
b
    clues = self._getclues(wordstream)
  File "/usr/local/lib/python2.4/site-packages/spambayes/classifier.py", line 498, in _getclues
    for word in Set(wordstream):
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py", line 1281, in tokenize
    for tok in self.tokenize_body(msg):
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py", line 1640, in tokenize_bod
y
    from spambayes.ImageStripper import crack_images
  File "/usr/local/lib/python2.4/site-packages/spambayes/ImageStripper.py", line 391, in ?
    crack_images = ImageStripper(_cachefile).analyze
  File "/usr/local/lib/python2.4/site-packages/spambayes/ImageStripper.py", line 305, in __init__
    self.cache = pickle.load(open(self.cachefile))
ValueError: insecure string pickle


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1816240&group_id=61702

From noreply at sourceforge.net  Fri Oct 19 19:29:38 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 19 Oct 2007 10:29:38 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1816240 ] General pickle
	corruption problems
Message-ID: <E1IivfG-0003WT-BE@sc8-sf-web24.sourceforge.net>

Bugs item #1816240, was opened at 2007-10-18 23:38
Message generated for change (Comment added) made by david_abrahams
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1816240&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: hammie
Group: Source code - CVS
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: David Abrahams (david_abrahams)
Assigned to: Nobody/Anonymous (nobody)
>Summary: General pickle corruption problems

Initial Comment:
saving 720 items to /home/dave/spambayes/imagecache.pck
Traceback (most recent call last):
  File "/usr/local/bin/sb_filter.py", line 290, in ?
    main()
  File "/usr/local/bin/sb_filter.py", line 281, in main
    action(msg)
  File "/usr/local/bin/sb_filter.py", line 199, in filter
    return self.h.filter(msg)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 156, in filter
    debug, train)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 110, in score_and_filter
    prob, clues = self._scoremsg(msg, True)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 39, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File "/usr/local/lib/python2.4/site-packages/spambayes/classifier.py", line 196, in chi2_spampro
b
    clues = self._getclues(wordstream)
  File "/usr/local/lib/python2.4/site-packages/spambayes/classifier.py", line 498, in _getclues
    for word in Set(wordstream):
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py", line 1281, in tokenize
    for tok in self.tokenize_body(msg):
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py", line 1640, in tokenize_bod
y
    from spambayes.ImageStripper import crack_images
  File "/usr/local/lib/python2.4/site-packages/spambayes/ImageStripper.py", line 391, in ?
    crack_images = ImageStripper(_cachefile).analyze
  File "/usr/local/lib/python2.4/site-packages/spambayes/ImageStripper.py", line 305, in __init__
    self.cache = pickle.load(open(self.cachefile))
ValueError: insecure string pickle


----------------------------------------------------------------------

>Comment By: David Abrahams (david_abrahams)
Date: 2007-10-19 12:29

Message:
Logged In: YES 
user_id=52572
Originator: YES

There is a safe method for storing pickles, but it isn't used
consistently, and there's code duplication.  The enclosed patch contains
fixes for the tte script and a few other things I've reported... the
dnscache changes are even related... but you *really* should apply the
whole thing ;-).
File Added: safe_pickle.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1816240&group_id=61702

From noreply at sourceforge.net  Mon Oct 22 04:29:44 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 21 Oct 2007 19:29:44 -0700
Subject: [spambayes-bugs] [ spambayes-Patches-1803130 ] Make tte ratio
	well-behaved when balance is good
Message-ID: <E1Ijn32-0008TW-UY@sc8-sf-web22.sourceforge.net>

Patches item #1803130, was opened at 2007-09-26 18:50
Message generated for change (Comment added) made by montanaro
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498105&aid=1803130&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: David Abrahams (david_abrahams)
>Assigned to: Skip Montanaro (montanaro)
Summary: Make tte ratio well-behaved when balance is good

Initial Comment:
Right now, if you pass a spam/ham ratio to tte and the training set is actually more balanced than indicated by the ratio, training will skip some messages in order to achieve that more-unbalanced ratio.  Nobody wants that.  The enclosed patch fixes it.

----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2007-10-21 21:29

Message:
Logged In: YES 
user_id=44345
Originator: NO

Checked in as contrib/tte.py r3102.  Thanks.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498105&aid=1803130&group_id=61702

From noreply at sourceforge.net  Mon Oct 22 04:32:21 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 21 Oct 2007 19:32:21 -0700
Subject: [spambayes-bugs] [ spambayes-Patches-1755857 ] tte.py Improved
	ratio handling
Message-ID: <E1Ijn5Z-0000LD-Ae@sc8-sf-web22.sourceforge.net>

Patches item #1755857, was opened at 2007-07-17 22:18
Message generated for change (Settings changed) made by montanaro
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498105&aid=1755857&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Duplicate
Priority: 5
Private: No
Submitted By: David Abrahams (david_abrahams)
Assigned to: Skip Montanaro (montanaro)
Summary: tte.py Improved ratio handling

Initial Comment:
If ham and spam are more balanced than the ratio argument (e.g. ratio is 2:1 and spam and ham counts are 150 and 100, the current behavior is to train on 150 spams and 75 hams.  That's really never what you want.  The ratio should delimit the unbalanced-ness.  The attached patch does that. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498105&aid=1755857&group_id=61702

From noreply at sourceforge.net  Mon Oct 22 04:35:46 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 21 Oct 2007 19:35:46 -0700
Subject: [spambayes-bugs] [ spambayes-Patches-1803128 ] Protection for
	dnscache corruption
Message-ID: <E1Ijn8s-0000fr-A3@sc8-sf-web22.sourceforge.net>

Patches item #1803128, was opened at 2007-09-26 18:43
Message generated for change (Comment added) made by montanaro
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498105&aid=1803128&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: David Abrahams (david_abrahams)
Assigned to: Nobody/Anonymous (nobody)
Summary: Protection for dnscache corruption

Initial Comment:
My spam filtering regularly starts failing because the dnscache is corrupted.  I don't have any idea how it gets that way, but a broken cache should never scuttle the process, so I propose the enclosed patch.

procmail: Executing "sb_filter.py"
Traceback (most recent call last):
  File "/usr/local/bin/sb_filter.py", line 83, in ?
    from spambayes import hammie, Options, mboxutils, storage
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 7, in ?
    from spambayes.tokenizer import tokenize
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py", line 44, in ?
    cache = dnscache.cache(cachefile=options["Tokenizer", "lookup_ip_cache"])
  File "/usr/local/lib/python2.4/site-packages/spambayes/dnscache.py", line 98, in __init__
    self.caches = pickle.load(open(self.cachefile, "rb"))
cPickle.UnpicklingError: invalid load key, '
'.
procmail: Program failure (1) of "sb_filter.py"


----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2007-10-21 21:35

Message:
Logged In: YES 
user_id=44345
Originator: NO

I'm trying to figure out what the patch to hammie.py has to do with
protecting against a corrupt
dnscache.  Was perhaps the wrong patch attached to this ticket?



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498105&aid=1803128&group_id=61702

From noreply at sourceforge.net  Mon Oct 22 04:47:39 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 21 Oct 2007 19:47:39 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1816240 ] General pickle
	corruption problems
Message-ID: <E1IjnKN-0005kE-R2@sc8-sf-web23.sourceforge.net>

Bugs item #1816240, was opened at 2007-10-18 23:38
Message generated for change (Comment added) made by montanaro
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1816240&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: hammie
>Group: 1.1.x
>Status: Pending
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: David Abrahams (david_abrahams)
>Assigned to: Skip Montanaro (montanaro)
Summary: General pickle corruption problems

Initial Comment:
saving 720 items to /home/dave/spambayes/imagecache.pck
Traceback (most recent call last):
  File "/usr/local/bin/sb_filter.py", line 290, in ?
    main()
  File "/usr/local/bin/sb_filter.py", line 281, in main
    action(msg)
  File "/usr/local/bin/sb_filter.py", line 199, in filter
    return self.h.filter(msg)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 156, in filter
    debug, train)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 110, in score_and_filter
    prob, clues = self._scoremsg(msg, True)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 39, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File "/usr/local/lib/python2.4/site-packages/spambayes/classifier.py", line 196, in chi2_spampro
b
    clues = self._getclues(wordstream)
  File "/usr/local/lib/python2.4/site-packages/spambayes/classifier.py", line 498, in _getclues
    for word in Set(wordstream):
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py", line 1281, in tokenize
    for tok in self.tokenize_body(msg):
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py", line 1640, in tokenize_bod
y
    from spambayes.ImageStripper import crack_images
  File "/usr/local/lib/python2.4/site-packages/spambayes/ImageStripper.py", line 391, in ?
    crack_images = ImageStripper(_cachefile).analyze
  File "/usr/local/lib/python2.4/site-packages/spambayes/ImageStripper.py", line 305, in __init__
    self.cache = pickle.load(open(self.cachefile))
ValueError: insecure string pickle


----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2007-10-21 21:47

Message:
Logged In: YES 
user_id=44345
Originator: NO

I don't understand what the hammie.py patch has to do with this.  I've
checked everything else
in.

----------------------------------------------------------------------

Comment By: David Abrahams (david_abrahams)
Date: 2007-10-19 12:29

Message:
Logged In: YES 
user_id=52572
Originator: YES

There is a safe method for storing pickles, but it isn't used
consistently, and there's code duplication.  The enclosed patch contains
fixes for the tte script and a few other things I've reported... the
dnscache changes are even related... but you *really* should apply the
whole thing ;-).
File Added: safe_pickle.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1816240&group_id=61702

From noreply at sourceforge.net  Mon Oct 22 04:52:27 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 21 Oct 2007 19:52:27 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1816240 ] General pickle
	corruption problems
Message-ID: <E1IjnP1-0007wz-Qu@sc8-sf-web24.sourceforge.net>

Bugs item #1816240, was opened at 2007-10-18 23:38
Message generated for change (Comment added) made by montanaro
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1816240&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: hammie
Group: 1.1.x
>Status: Closed
Resolution: Accepted
Priority: 5
Private: No
Submitted By: David Abrahams (david_abrahams)
Assigned to: Skip Montanaro (montanaro)
Summary: General pickle corruption problems

Initial Comment:
saving 720 items to /home/dave/spambayes/imagecache.pck
Traceback (most recent call last):
  File "/usr/local/bin/sb_filter.py", line 290, in ?
    main()
  File "/usr/local/bin/sb_filter.py", line 281, in main
    action(msg)
  File "/usr/local/bin/sb_filter.py", line 199, in filter
    return self.h.filter(msg)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 156, in filter
    debug, train)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 110, in score_and_filter
    prob, clues = self._scoremsg(msg, True)
  File "/usr/local/lib/python2.4/site-packages/spambayes/hammie.py", line 39, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File "/usr/local/lib/python2.4/site-packages/spambayes/classifier.py", line 196, in chi2_spampro
b
    clues = self._getclues(wordstream)
  File "/usr/local/lib/python2.4/site-packages/spambayes/classifier.py", line 498, in _getclues
    for word in Set(wordstream):
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py", line 1281, in tokenize
    for tok in self.tokenize_body(msg):
  File "/usr/local/lib/python2.4/site-packages/spambayes/tokenizer.py", line 1640, in tokenize_bod
y
    from spambayes.ImageStripper import crack_images
  File "/usr/local/lib/python2.4/site-packages/spambayes/ImageStripper.py", line 391, in ?
    crack_images = ImageStripper(_cachefile).analyze
  File "/usr/local/lib/python2.4/site-packages/spambayes/ImageStripper.py", line 305, in __init__
    self.cache = pickle.load(open(self.cachefile))
ValueError: insecure string pickle


----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2007-10-21 21:52

Message:
Logged In: YES 
user_id=44345
Originator: NO

I went ahead and checked in the change to hammie.py.  I can see that if
you opened the file for reading that you won't want to store it later.



----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-10-21 21:47

Message:
Logged In: YES 
user_id=44345
Originator: NO

I don't understand what the hammie.py patch has to do with this.  I've
checked everything else
in.

----------------------------------------------------------------------

Comment By: David Abrahams (david_abrahams)
Date: 2007-10-19 12:29

Message:
Logged In: YES 
user_id=52572
Originator: YES

There is a safe method for storing pickles, but it isn't used
consistently, and there's code duplication.  The enclosed patch contains
fixes for the tte script and a few other things I've reported... the
dnscache changes are even related... but you *really* should apply the
whole thing ;-).
File Added: safe_pickle.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1816240&group_id=61702

From noreply at sourceforge.net  Tue Oct 23 07:59:51 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 22 Oct 2007 22:59:51 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1818324 ] Junk E-Mail directory
	disappears
Message-ID: <E1IkCnv-0007lx-7t@sc8-sf-web21.sourceforge.net>

Bugs item #1818324, was opened at 2007-10-23 07:59
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1818324&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Outlook
Group: 1.0.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: avi909 (avi909)
Assigned to: Nobody/Anonymous (nobody)
Summary: Junk E-Mail directory disappears

Initial Comment:
Junk E-Mail directory disappears. Deleting as a Spam cused erasion of the message.
I tried to build a new directory with the same name. No spam was send to the new directory and messages deleteed as a spam disappeared. 
I reinstalled SpamBayes 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1818324&group_id=61702

From noreply at sourceforge.net  Thu Oct 25 12:59:48 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 25 Oct 2007 03:59:48 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1819910 ] AttributeError,
	get_statistics_start_date
Message-ID: <E1Il0RI-0006Uw-Vs@sc8-sf-web22.sourceforge.net>

Bugs item #1819910, was opened at 2007-10-25 10:59
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1819910&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: pop3proxy
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Peter Bengtsson (peterbe)
Assigned to: Nobody/Anonymous (nobody)
Summary: AttributeError, get_statistics_start_date

Initial Comment:
python2.5 in Ubuntu I start sb_server and just enter the remove server and remote proxy and press the Save button. Then I get this error:


500 Server error

Traceback (most recent call last):

  File "/usr/lib/python2.5/site-packages/spambayes/Dibbler.py", line 476, in found_terminator
    getattr(plugin, name)(**params)

  File "/usr/lib/python2.5/site-packages/spambayes/UserInterface.py", line 888, in onChangeopts
    self.reReadOptions()

  File "/usr/lib/python2.5/site-packages/spambayes/ProxyUI.py", line 587, in reReadOptions
    state = self.state_recreator()

  File "/usr/bin/sb_server.py", line 1007, in _recreateState
    prepare()

  File "/usr/bin/sb_server.py", line 1022, in prepare
    state.prepare(can_stop)

  File "/usr/bin/sb_server.py", line 822, in prepare
    self.createWorkers()

  File "/usr/bin/sb_server.py", line 889, in createWorkers
    self.stats = Stats.Stats(options, self.mdb)

  File "/usr/lib/python2.5/site-packages/spambayes/Stats.py", line 60, in __init__
    self.from_date = self.messageinfo_db.get_statistics_start_date()

AttributeError: 'NoneType' object has no attribute 'get_statistics_start_date'


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1819910&group_id=61702

From noreply at sourceforge.net  Thu Oct 25 13:09:36 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 25 Oct 2007 04:09:36 -0700
Subject: [spambayes-bugs] [ spambayes-Bugs-1819910 ] AttributeError,
	get_statistics_start_date
Message-ID: <E1Il0am-00026X-JA@sc8-sf-web24.sourceforge.net>

Bugs item #1819910, was opened at 2007-10-25 10:59
Message generated for change (Comment added) made by peterbe
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1819910&group_id=61702

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: pop3proxy
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Peter Bengtsson (peterbe)
Assigned to: Nobody/Anonymous (nobody)
Summary: AttributeError, get_statistics_start_date

Initial Comment:
python2.5 in Ubuntu I start sb_server and just enter the remove server and remote proxy and press the Save button. Then I get this error:


500 Server error

Traceback (most recent call last):

  File "/usr/lib/python2.5/site-packages/spambayes/Dibbler.py", line 476, in found_terminator
    getattr(plugin, name)(**params)

  File "/usr/lib/python2.5/site-packages/spambayes/UserInterface.py", line 888, in onChangeopts
    self.reReadOptions()

  File "/usr/lib/python2.5/site-packages/spambayes/ProxyUI.py", line 587, in reReadOptions
    state = self.state_recreator()

  File "/usr/bin/sb_server.py", line 1007, in _recreateState
    prepare()

  File "/usr/bin/sb_server.py", line 1022, in prepare
    state.prepare(can_stop)

  File "/usr/bin/sb_server.py", line 822, in prepare
    self.createWorkers()

  File "/usr/bin/sb_server.py", line 889, in createWorkers
    self.stats = Stats.Stats(options, self.mdb)

  File "/usr/lib/python2.5/site-packages/spambayes/Stats.py", line 60, in __init__
    self.from_date = self.messageinfo_db.get_statistics_start_date()

AttributeError: 'NoneType' object has no attribute 'get_statistics_start_date'


----------------------------------------------------------------------

>Comment By: Peter Bengtsson (peterbe)
Date: 2007-10-25 11:09

Message:
Logged In: YES 
user_id=208160
Originator: YES

Tried to downgrade to 1.1a3 but problem persists :(

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1819910&group_id=61702