Scanning a file character by character

[Tim Chase]

Josh Dukes wrote:
> In [401]: import shlex
> 
> In [402]: shlex.split("""Joe went to 'the store' where he bought a "box of chocolates" and stuff.""")
> 
> how's that work for ya? 

It works great if that's the desired behavior.  However, the OP 
wrote about splitting the lines into separate words, not 
"treating quoted items as a single word".  (OP: "How would I do 
separate lines into words without scanning one character at a time?")

But for pulling out quoted strings as units, the shlex is a great 
module.

-tkc