[spambayes-bugs] [ spambayes-Bugs-922063 ] Intermittent sb_filter.py failure with URL pickle

[SourceForge.net]

Bugs item #922063, was opened at 2004-03-24 10:10
Message generated for change (Comment added) made by anadelonbrin
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=922063&group_id=61702

Category: hammie
>Group: Source code - CVS
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Leonid (leobru)
Assigned to: Tony Meyer (anadelonbrin)
>Summary: Intermittent sb_filter.py failure with URL pickle

Initial Comment:
Here are the relevant .spambayesrc lines:

[Tokenizer]
x-fancy_url_recognition=True
x-pick_apart_urls=True

[URLRetriever]
x-slurp_urls=True

Here is the stack trace:

  File
"/usr/home/leob/spambayes-1.0a9/scripts/sb_filter.py",
line 239, in ?
    main()
  File
"/usr/home/leob/spambayes-1.0a9/scripts/sb_filter.py",
line 231, in main
    action(msg)
  File
"/usr/home/leob/spambayes-1.0a9/scripts/sb_filter.py",
line 163, in filter
    return h.filter(msg)
  File
"/usr/home/leob/opt/lib/python2.2/site-packages/spambayes/hammie.py",
line 109, in filter
    prob, clues = self._scoremsg(msg, True)
  File
"/usr/home/leob/opt/lib/python2.2/site-packages/spambayes/hammie.py",
line 38, in _scoremsg
    return self.bayes.spamprob(tokenize(msg), evidence)
  File
"/usr/home/leob/opt/lib/python2.2/site-packages/spambayes/classifier.py",
line 246, in slurpi
ng_spamprob
    slurp_tokens = list(self._generate_slurp())
  File
"/usr/home/leob/opt/lib/python2.2/site-packages/spambayes/classifier.py",
line 550, in _gener
ate_slurp
    self.setup()
  File
"/usr/home/leob/opt/lib/python2.2/site-packages/spambayes/classifier.py",
line 609, in setup
    self.bad_urls = pickle.load(b_file)
  File "/usr/home/leob/opt/lib/python2.2/pickle.py",
line 982, in load
    return Unpickler(file).load()
  File "/usr/home/leob/opt/lib/python2.2/pickle.py",
line 597, in load
    dispatch[key](self)
  File "/usr/home/leob/opt/lib/python2.2/pickle.py",
line 667, in load_string
    raise ValueError, "insecure string pickle"


----------------------------------------------------------------------

>Comment By: Tony Meyer (anadelonbrin)
Date: 2004-11-03 14:11

Message:
Logged In: YES 
user_id=552329

For the sake of resolving this, I've changed the code anyway:

 1. If an error occurs loading the pickle, then a new one is
used - at least the classifier will keep going, and this
shouldn't hurt much (it's only a cache).

 2. Saving saves to a temp file first, and then replaces the
old one.  This should be completely (*nix) or reasonably
(win32) robust.

----------------------------------------------------------------------

Comment By: Tony Meyer (anadelonbrin)
Date: 2004-08-03 17:59

Message:
Logged In: YES 
user_id=552329

I'm guessing that something went wrong writing the pickle. 
(I get an EOFError trying to open the attached pickle).  The
slurping code really ought to do what the other code does
and save a copy and then replace the original once the save
succeeds.

I'm reluctant to do this at the moment, though, since it
seems fairly likely that the slurping code will vanish given
that it's only experimental and no-one's spoken up saying
that it does them any good.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=922063&group_id=61702