Tokenizing a string

[Moshe Zadka]

On Sat, 18 Mar 2000, Michael Dartt wrote:

> I've got a string I'd like to tokenize, but it's not in a file, and it'd
> be rather inefficient to write it to a file just to tokenize it.  Is
> there any function I can use to pass this string to
> tokenize.tokenize()?  

Use StringIO or cStringIO, or just use split('\n') on it, and write a
small class to get out the lines.

--
Moshe Zadka <mzadka at geocities.com>. 
http://www.oreilly.com/news/prescod_0300.html
http://www.linux.org.il -- we put the penguin in .com