I have used pywinauto for such tasks in the past. http://pywinauto.openqa.org/ In my case, I used pywinauto to automate mouse clicks on a browser in order to auto-play a Flash game running in the browser. I had to use PIL to take screenshots and then process images to "read" the screen. -- Paul