[Spambayes-checkins] spambayes tokenizer.py,1.59,1.60

[Sjoerd Mullender]

Update of /cvsroot/spambayes/spambayes
In directory usw-pr-cvs1:/tmp/cvs-serv13555

Modified Files:
	tokenizer.py 
Log Message:
Switch " and ' in url_re character class and add # ' token the re to
resync python-mode.


Index: tokenizer.py
===================================================================
RCS file: /cvsroot/spambayes/spambayes/tokenizer.py,v
retrieving revision 1.59
retrieving revision 1.60
diff -C2 -d -r1.59 -r1.60
*** tokenizer.py	31 Oct 2002 15:43:55 -0000	1.59
--- tokenizer.py	1 Nov 2002 16:10:13 -0000	1.60
***************
*** 604,609 ****
      # be in HTML, may or may not be in quotes, etc.  If it's full of %
      # escapes, cool -- that's a clue too.
!     ([^\s<>'"\x7f-\xff]+)  # capture the guts
! """, re.VERBOSE)
  
  urlsep_re = re.compile(r"[;?:@&=+,$.]")
--- 604,609 ----
      # be in HTML, may or may not be in quotes, etc.  If it's full of %
      # escapes, cool -- that's a clue too.
!     ([^\s<>"'\x7f-\xff]+)  # capture the guts
! """, re.VERBOSE)                        # '
  
  urlsep_re = re.compile(r"[;?:@&=+,$.]")