Good String Tokenizer

[Duncan Booth]

JamesHoward <James.w.Howard at gmail.com> wrote:

> I have searched the board
what board? I don't see any boards here.

> 
> And my splitting points are comma, and exclamation point then I would
> expect to get back.
> 
> ["Hello", ",", " World", "!", "  How are you?"]
> 
> Does anyone know of a tokenizer that will allow for this sort of use?
> 

>>> import re
>>> re.split("([!,])", "Hello, World!  How are you?")
['Hello', ',', ' World', '!', '  How are you?']