incorrect(?) shlex behaviour

[bill]

Its gets worse:
>>> from shlex import StringIO
>>> from shlex import shlex
>>> t = shlex(StringIO("2>&1"))
>>> while True:
...  b = t.read_token()
...  if not b: break
...  print b
...
2
&
1                    <----------- where's the '>' !?
>>> import shlex
>>> print shlex.split("2>&1")
['2>&1']

It strikes me that split should be behaving exactly the same way as
read_token, but that may be a misunderstanding on my part of what split
is doing.

However, it is totally bizarre that read_token discards the '>' symbol
in the string!  I don't know much about lexical analysis, but it
strikes me that discarding characters is a bad thing.