My first attempt at python

[Mike Fletcher]

Minor note, you might want to make the tokeniser use RE's
"word"/"whitespace" abstract character class to make the tokenising a little
more robust:

>>> import re
>>> re.split( '\W+', 'this. that() is a {} test',)
['this', 'that', 'is', 'a', 'test']

Enjoy,
Mike

-----Original Message-----
From: David Porter [mailto:jcm at bigskytel.com]
Sent: Friday, December 08, 2000 8:30 AM
To: Jason Stewart
Cc: Python List
Subject: Re: My first attempt at python
...
import re
def tokenize(s):
    r = re.compile(' ')
    return r.split(s)
...
Of course, the best way is:

import string
tokens = string.split(s)
...