Could this expression parser be more 'Pythonic'?

[Amr]

Hello all,

I've been spending the last few weeks learning Python, and I've just
started to use it to write a simple BASIC compiler. I'm writing a
mathematical expression parser and wrote a function that would take a
string and split it into high level tokens.

The code can be found at http://pastebin.com/f757252ec (have a look at
the function tokenize).

I was interested to see if the tokenize function could be written in a
better way, so as to make better use of the Python language and
libraries. One thing that sprung to mind was whether it would be a
good idea to perhaps take advantage of regular expressions. I had a
look at a few compiler sites and regexps seem to come into that role
quite a bit. I've only done a little work on regexps and they seem
very powerful, but can easily get out of control!

So, any suggestions as to how to make better use of the language
writing something like this?

Thanks,

--Amr