From skip at pobox.com Wed Jun 17 13:50:46 2009 From: skip at pobox.com (skip at pobox.com) Date: Wed, 17 Jun 2009 06:50:46 -0500 Subject: [spambayes-dev] [Spambayes] What does this mean? In-Reply-To: <4A38696C.6080502@indicium.org> References: <4A38696C.6080502@indicium.org> Message-ID: <19000.55446.705124.770738@montanaro.dyndns.org> Eric> File "/usr/lib/python2.6/asyncore.py", line 391, in __getattr__ Eric> return getattr(self.socket, attr) Eric> AttributeError: '_socketobject' object has no attribute Eric> 'ac_out_buffer' Eric> This is Spambayes 1.1a4 installed on a fresh install of Ubuntu Eric> 9.04. Eric> Could someone explain to me what I broke? You didn't break anything. The ground under your feet shifted. In Python 2.6 someone reworked the asyncore and asynchat modules in backwards-incompatible ways. One of the things that was done was to remove that ac_out_buffer attribute. Unfortunately, it looks like Dibbler.py uses it. This whole episode caused quite a stir on python-dev when it came to light. (Not enough, apparently, to restore the code in 2.7 though.) I'm not certain of the correct fix, not being very familiar with the changes in Python 2.6 or with the Dibbler module. Here are a couple things you might try: * Install Python 2.5 on your system and use that instead of Python 2.6. * Modify your copy of Dibbler.py to remove the (sole) reference to self.ac_out_buffer. Change: while (self.producer_fifo or self.ac_out_buffer) and not self._closed: to acob = (hasattr(self, "ac_out_buffer") and self.ac_out_buffer or True) while (self.producer_fifo or acob) and not self._closed: * Staring at the thread on python-dev suggests that this might work as well: while self.writable() and not self._closed: I'm pretty sure switching to Python 2.5 will work, though that may cause your system some consternation if you have other packages which require 2.6 and you can't install both of them at the same time (I suspect that will be ok). I'm a bit less sure about the second and third changes, but I think they should work. Clearly the third is clearer and likely more correct as well. Full details on the change from python-dev: http://news.gmane.org/find-root.php?message_id=%3c48EE2C77.1050809%40palladion.com%3e Richie, care to chime in? -- Skip Montanaro - skip at pobox.com - http://www.smontanaro.net/ when i wake up with a heart rate below 40, i head right for the espresso machine. -- chaos @ forums.usms.org From richie at entrian.com Thu Jun 18 00:00:18 2009 From: richie at entrian.com (Richie Hindle) Date: Wed, 17 Jun 2009 23:00:18 +0100 Subject: [spambayes-dev] [Spambayes] What does this mean? In-Reply-To: <19000.55446.705124.770738@montanaro.dyndns.org> References: <4A38696C.6080502@indicium.org> <19000.55446.705124.770738@montanaro.dyndns.org> Message-ID: <8a6ba1da0906171500p7289a4d3u9968f6641388b265@mail.gmail.com> [Skip] > Richie, care to chime in? Eric, Skip's advice to use Python 2.5 is your best bet. There's no reason you can't install it alongside other versions, and I believe there's an option in the installer to not change your file associations etc. (and if there isn't, you can always reinstall the later version afterwards). Skip, I'll have a look at your proposed fixes this week, but I don't have time tonight - it's bed time here. 8-) -- Richie Hindle richie at entrian.com From ej.spambayes at indicium.org Thu Jun 18 00:46:18 2009 From: ej.spambayes at indicium.org (Eric Johnson) Date: Wed, 17 Jun 2009 16:46:18 -0600 Subject: [spambayes-dev] [Spambayes] What does this mean? In-Reply-To: <8a6ba1da0906171500p7289a4d3u9968f6641388b265@mail.gmail.com> References: <4A38696C.6080502@indicium.org> <19000.55446.705124.770738@montanaro.dyndns.org> <8a6ba1da0906171500p7289a4d3u9968f6641388b265@mail.gmail.com> Message-ID: <4A39723A.3030603@indicium.org> On 09-06-17 04:00 PM, Richie Hindle wrote: > [Skip] >> Richie, care to chime in? > > Eric, Skip's advice to use Python 2.5 is your best bet. There's no > reason you can't install it alongside other versions, and I believe > there's an option in the installer to not change your file > associations etc. (and if there isn't, you can always reinstall the > later version afterwards). > > Skip, I'll have a look at your proposed fixes this week, but I don't > have time tonight - it's bed time here. 8-) > I checked Synaptic to what in regards to Python was installed. It says that Python 2.5 and 2.6 are installed. Is it possible to specify in the command line which version of Python to use? Thanks, Eric From skip at pobox.com Thu Jun 18 03:06:06 2009 From: skip at pobox.com (skip at pobox.com) Date: Wed, 17 Jun 2009 20:06:06 -0500 Subject: [spambayes-dev] [Spambayes] What does this mean? In-Reply-To: <4A39723A.3030603@indicium.org> References: <4A38696C.6080502@indicium.org> <19000.55446.705124.770738@montanaro.dyndns.org> <8a6ba1da0906171500p7289a4d3u9968f6641388b265@mail.gmail.com> <4A39723A.3030603@indicium.org> Message-ID: <19001.37630.441641.399207@montanaro.dyndns.org> Eric> I checked Synaptic to what in regards to Python was installed. It Eric> says that Python 2.5 and 2.6 are installed. Eric> Is it possible to specify in the command line which version of Eric> Python to use? I sort of assume you installed SpamBayes something like this: python setup.py install You can refrence Python 2.5 directly: python2.5 setup.py install SpamBayes will then be run using Python 2.5. If you install SpamBayes using Synaptics, I don't know how to tell it to use Python 2.5. Skip From ej.spambayes at indicium.org Thu Jun 18 03:54:07 2009 From: ej.spambayes at indicium.org (Eric Johnson) Date: Wed, 17 Jun 2009 19:54:07 -0600 Subject: [spambayes-dev] [Spambayes] What does this mean? In-Reply-To: <19001.37630.441641.399207@montanaro.dyndns.org> References: <4A38696C.6080502@indicium.org> <19000.55446.705124.770738@montanaro.dyndns.org> <8a6ba1da0906171500p7289a4d3u9968f6641388b265@mail.gmail.com> <4A39723A.3030603@indicium.org> <19001.37630.441641.399207@montanaro.dyndns.org> Message-ID: <4A399E3F.2040606@indicium.org> On 09-06-17 07:06 PM, skip at pobox.com wrote: > Eric> I checked Synaptic to what in regards to Python was installed. It > Eric> says that Python 2.5 and 2.6 are installed. > > Eric> Is it possible to specify in the command line which version of > Eric> Python to use? > > I sort of assume you installed SpamBayes something like this: > > python setup.py install > > You can refrence Python 2.5 directly: > > python2.5 setup.py install > > SpamBayes will then be run using Python 2.5. If you install SpamBayes using > Synaptics, I don't know how to tell it to use Python 2.5. > > Skip > > . > I tried your suggestion of using "python2.5 setup.py install". I launch the proxy this way, "python2.5 sb_server.py" I am no longer getting an error message when training, but I am now getting this error message when I am using the configuration pages... 500 Server error Traceback (most recent call last): File "/usr/lib/python2.5/site-packages/spambayes/Dibbler.py", line 476, in found_terminator getattr(plugin, name)(**params) File "/usr/lib/python2.5/site-packages/spambayes/UserInterface.py", line 888, in onChangeopts self.reReadOptions() File "/usr/lib/python2.5/site-packages/spambayes/ProxyUI.py", line 587, in reReadOptions state = self.state_recreator() File "./spambayes/scripts/sb_server.py", line 1007, in _recreateState prepare() File "./spambayes/scripts/sb_server.py", line 1022, in prepare state.prepare(can_stop) File "./spambayes/scripts/sb_server.py", line 822, in prepare self.createWorkers() File "./spambayes/scripts/sb_server.py", line 889, in createWorkers self.stats = Stats.Stats(options, self.mdb) File "/usr/lib/python2.5/site-packages/spambayes/Stats.py", line 60, in __init__ self.from_date = self.messageinfo_db.get_statistics_start_date() AttributeError: 'NoneType' object has no attribute 'get_statistics_start_date' Is this something different or part of the same problem as before? Thanks for all the suggestions, Eric From skip at pobox.com Thu Jun 18 11:42:50 2009 From: skip at pobox.com (skip at pobox.com) Date: Thu, 18 Jun 2009 04:42:50 -0500 Subject: [spambayes-dev] [Spambayes] What does this mean? In-Reply-To: <4A399E3F.2040606@indicium.org> References: <4A38696C.6080502@indicium.org> <19000.55446.705124.770738@montanaro.dyndns.org> <8a6ba1da0906171500p7289a4d3u9968f6641388b265@mail.gmail.com> <4A39723A.3030603@indicium.org> <19001.37630.441641.399207@montanaro.dyndns.org> <4A399E3F.2040606@indicium.org> Message-ID: <19002.3098.305856.221380@montanaro.dyndns.org> Eric> I tried your suggestion of using "python2.5 setup.py install". Eric> I launch the proxy this way, "python2.5 sb_server.py" That works, but you're running SpamBayes from your source repository, not from the installed location. Try running /usr/local/bin/sb_server.py instead, or if /usr/local/bin is in your PATH, simply sb_server.py Eric> I am no longer getting an error message when training, but I am now Eric> getting this error message when I am using the configuration pages... Eric> 500 Server error ... Eric> File "./spambayes/scripts/sb_server.py", line 889, in createWorkers Eric> self.stats = Stats.Stats(options, self.mdb) Eric> File "/usr/lib/python2.5/site-packages/spambayes/Stats.py", line 60, Eric> in __init__ Eric> self.from_date = self.messageinfo_db.get_statistics_start_date() Eric> AttributeError: 'NoneType' object has no attribute Eric> 'get_statistics_start_date' Eric> Is this something different or part of the same problem as before? This is something new. Are you trying to train before you're done configuring the setup? I wouldn't think you'd be creating a Stats object (the beginning of what I quoted) at that point. I'm not terribly familiar with the POP3 proxy (sb_server) anymore, and it will probably be a day or two before I have a chance to take a look at the problem. In the meantime, double-check that you've completely configured the server before trying to do any training. Skip From dave at boostpro.com Fri Jun 19 14:48:44 2009 From: dave at boostpro.com (David Abrahams) Date: Fri, 19 Jun 2009 08:48:44 -0400 Subject: [spambayes-dev] spambayes.sf.net down? Message-ID: FYI, it looks like the Spambayes website is offline. -- Dave Abrahams BoostPro Computing http://www.boostpro.com From tim.peters at gmail.com Fri Jun 19 15:01:45 2009 From: tim.peters at gmail.com (Tim Peters) Date: Fri, 19 Jun 2009 09:01:45 -0400 Subject: [spambayes-dev] spambayes.sf.net down? In-Reply-To: References: Message-ID: <1f7befae0906190601y784dd496rc9405a7ea57b7c5@mail.gmail.com> [David Abrahams] > FYI, it looks like the Spambayes website is offline. Trying http://spambayes.sf.net/ worked fine for me just now. If it ever is down, chances are high only someone at SourceForge could explain why (or repair it). From skip at pobox.com Fri Jun 19 15:56:01 2009 From: skip at pobox.com (skip at pobox.com) Date: Fri, 19 Jun 2009 08:56:01 -0500 Subject: [spambayes-dev] spambayes.sf.net down? In-Reply-To: <1f7befae0906190601y784dd496rc9405a7ea57b7c5@mail.gmail.com> References: <1f7befae0906190601y784dd496rc9405a7ea57b7c5@mail.gmail.com> Message-ID: <19003.39153.698759.477710@montanaro.dyndns.org> Tim> [David Abrahams] >> FYI, it looks like the Spambayes website is offline. Tim> Trying Tim> http://spambayes.sf.net/ Tim> worked fine for me just now. Tim> If it ever is down, chances are high only someone at SourceForge could Tim> explain why (or repair it). Agreed. Not much we can do. I got a Connection Interrupted message from FF just now. Skip From dave at boostpro.com Fri Jun 19 16:11:06 2009 From: dave at boostpro.com (David Abrahams) Date: Fri, 19 Jun 2009 10:11:06 -0400 Subject: [spambayes-dev] spambayes.sf.net down? In-Reply-To: <1f7befae0906190601y784dd496rc9405a7ea57b7c5@mail.gmail.com> (Tim Peters's message of "Fri, 19 Jun 2009 09:01:45 -0400") References: <1f7befae0906190601y784dd496rc9405a7ea57b7c5@mail.gmail.com> Message-ID: on Fri Jun 19 2009, Tim Peters wrote: > [David Abrahams] >> FYI, it looks like the Spambayes website is offline. > > Trying > > http://spambayes.sf.net/ > > worked fine for me just now. > > If it ever is down, chances are high only someone at SourceForge could > explain why (or repair it). It's something in the network. Connection Interrupted The connection to the server was reset while the page was loading. The network link was interrupted while negotiating a connection. Please try again. I don't seem to be able to reach any .sf.net site. Sorry for the noise, -- Dave Abrahams BoostPro Computing http://www.boostpro.com From ej.spambayes at indicium.org Sat Jun 20 04:37:44 2009 From: ej.spambayes at indicium.org (Eric Johnson) Date: Fri, 19 Jun 2009 20:37:44 -0600 Subject: [spambayes-dev] [Spambayes] What does this mean? In-Reply-To: <19002.3098.305856.221380@montanaro.dyndns.org> References: <4A38696C.6080502@indicium.org> <19000.55446.705124.770738@montanaro.dyndns.org> <8a6ba1da0906171500p7289a4d3u9968f6641388b265@mail.gmail.com> <4A39723A.3030603@indicium.org> <19001.37630.441641.399207@montanaro.dyndns.org> <4A399E3F.2040606@indicium.org> <19002.3098.305856.221380@montanaro.dyndns.org> Message-ID: <4A3C4B78.2000801@indicium.org> On 09-06-18 03:42 AM, skip at pobox.com wrote: > Eric> I tried your suggestion of using "python2.5 setup.py install". > > Eric> I launch the proxy this way, "python2.5 sb_server.py" > > That works, but you're running SpamBayes from your source repository, not > from the installed location. Try running > > /usr/local/bin/sb_server.py > > instead, or if /usr/local/bin is in your PATH, simply > > sb_server.py > > Eric> I am no longer getting an error message when training, but I am now > Eric> getting this error message when I am using the configuration pages... > > Eric> 500 Server error > ... > Eric> File "./spambayes/scripts/sb_server.py", line 889, in createWorkers > Eric> self.stats = Stats.Stats(options, self.mdb) > > Eric> File "/usr/lib/python2.5/site-packages/spambayes/Stats.py", line 60, > Eric> in __init__ > Eric> self.from_date = self.messageinfo_db.get_statistics_start_date() > > Eric> AttributeError: 'NoneType' object has no attribute > Eric> 'get_statistics_start_date' > > Eric> Is this something different or part of the same problem as before? > > This is something new. Are you trying to train before you're done > configuring the setup? I wouldn't think you'd be creating a Stats object > (the beginning of what I quoted) at that point. > > I'm not terribly familiar with the POP3 proxy (sb_server) anymore, and it > will probably be a day or two before I have a chance to take a look at the > problem. In the meantime, double-check that you've completely configured > the server before trying to do any training. > > Skip > > . > Thanks for letting me know about the proper path to run the server from, I'm still getting used to Linux... What was happening when I was getting that error, was that I was trying to configure the proxy and when I would tell it to save the settings, that error message would show up and then I would have to restart the proxy to be able to do the rest of the configuration. The funny part of the proxy crashing during configuration is that when I restart it and go to check the settings, all the settings I wanted, were in fact saved. Anyway, I do still get the error message... 500 Server error Traceback (most recent call last): File "/usr/lib/python2.5/site-packages/spambayes/Dibbler.py", line 476, in found_terminator getattr(plugin, name)(**params) File "/usr/lib/python2.5/site-packages/spambayes/UserInterface.py", line 888, in onChangeopts self.reReadOptions() File "/usr/lib/python2.5/site-packages/spambayes/ProxyUI.py", line 587, in reReadOptions state = self.state_recreator() File "/usr/local/bin/sb_server.py", line 1007, in _recreateState prepare() File "/usr/local/bin/sb_server.py", line 1022, in prepare state.prepare(can_stop) File "/usr/local/bin/sb_server.py", line 822, in prepare self.createWorkers() File "/usr/local/bin/sb_server.py", line 889, in createWorkers self.stats = Stats.Stats(options, self.mdb) File "/usr/lib/python2.5/site-packages/spambayes/Stats.py", line 60, in __init__ self.from_date = self.messageinfo_db.get_statistics_start_date() AttributeError: 'NoneType' object has no attribute 'get_statistics_start_date' ...whenever the save button is pressed on any of the 3 configuration pages. Otherwise the proxy works fine and no problem with the "review" page or when training. I must say that this spam filter is the best one that I have ever come across, I just wish the others could be 25% as good. With training on as few as 12 ham and 12 spam, I am already getting as few as 0.3% unsure. Thanks, Eric From marko at von-oppen.com Mon Jun 22 11:31:16 2009 From: marko at von-oppen.com (Marko von Oppen) Date: Mon, 22 Jun 2009 11:31:16 +0200 Subject: [spambayes-dev] SpamBayes on Debian Message-ID: <93d157497257a1b71a6cb42a41229a4c@von-oppen.com> At beginning of the year after 15 years using MS Mail/Outlook I ported my E-Mail system to a server based setup using Dovecot IMAP and Roundcube Webmail and/or Apple Mail client. I'm using Debian Stable on the server. 3 months ago I installed Spambayes on the server using MySQL as storage backend. Here are some remarks and questions. - The Debian stable package 1.0.4 is really old. Because of an error in this version in the MySQL connect code SpamBayes/MySQL can not work. - When I looked into Debian Unstable I discovered that there the same old version of SpamBayes is in. Here for the first time the question came up: Is SpamBayes on Debian still maintained? Is the maintainer here on the list? The last entry in Debian changelog is a 'non-maintainer-upload'. - After fixing the MySQL connect bug and running the initial training the next problem came up: "assert spamcount <= nspam, "Token seen in more spam than spam trained."" - After that I decided to use newer code. Building a Debian package from the 1.1a4 code was a thing of around 15 minutes. The code from that package was running for around 8 weeks very stable. - But last week the error came back (see below). It seems that this bug is known to the developers? It is fixed in a newer version? - Next problem: Building a Debian package from the SVN trunk fails while it works fine in the same environment for 1.1a4. Does somebody understand what is causing this error? Thanks Marko MARKO VON OPPEN - TECHNISCHE SOFTWARE Ringstr. 3, 71691 Freiberg am Neckar, Germany fon +49 (0)7141 6080813, fax +49 (0)7141 6080814 e-mail marko at von-oppen.com web http://www.von-oppen.com/ === error report 1 === marko at mvob:~$ sb_filter.py -f main() File "/usr/bin/sb_filter.py", line 274, in main action(msg) File "/usr/bin/sb_filter.py", line 192, in filter return self.h.filter(msg) File "/usr/lib/python2.5/site-packages/spambayes/hammie.py", line 155, in filter debug, train) File "/usr/lib/python2.5/site-packages/spambayes/hammie.py", line 109, in score_and_filter prob, clues = self._scoremsg(msg, True) File "/usr/lib/python2.5/site-packages/spambayes/hammie.py", line 38, in _scoremsg return self.bayes.spamprob(tokenize(msg), evidence) File "/usr/lib/python2.5/site-packages/spambayes/classifier.py", line 196, in chi2_spamprob clues = self._getclues(wordstream) File "/usr/lib/python2.5/site-packages/spambayes/classifier.py", line 499, in _getclues tup = self._worddistanceget(word) File "/usr/lib/python2.5/site-packages/spambayes/classifier.py", line 514, in _worddistanceget prob = self.probability(record) File "/usr/lib/python2.5/site-packages/spambayes/classifier.py", line 317, in probability assert spamcount <= nspam, "Token seen in more spam than spam trained." AssertionError: Token seen in more spam than spam trained. mysql> select * from bayes order by nspam desc limit 10; +----------------------+-------+-------+ | word | nspam | nham | +----------------------+-------+-------+ | header:From:1 | 21835 | 26555 | | header:Subject:1 | 21830 | 26666 | | saved state | 21827 | 26807 | | header:To:1 | 21803 | 26422 | | header:Return-Path:1 | 21794 | 19516 | === Error while building Debian package from SVN sources === I tried to build the package using dpkg-buildpackage but the following is the command that fails: marko at mais:~/src/spambayes-1.1b1$ python setup.py install --prefix=debian/spambayes/usr --no-compile running install Checking .pth file support in debian/spambayes/usr/lib/python2.5/site-packages/ error: can't create or remove files in install directory The following error occurred while trying to add or remove files in the installation directory: [Errno 2] No such file or directory: 'debian/spambayes/usr/lib/python2.5/site-packages/test-easy-install-12399.pth' The installation directory you specified (via --install-dir, --prefix, or the distutils default setting) was: debian/spambayes/usr/lib/python2.5/site-packages/ This directory does not currently exist. Please create it and try again, or choose a different installation directory (using the -d or --install-dir option).