[python-win32] recording GUI events for playback (Java application)

[Tim Roberts]

tim.fulcher at bt.com wrote:
>  
> I’m looking to automate driving a sequence of steps of an application
> running on windows. I’m looking at pywinauto as a possible mechanism
> which maybe can do what I need. But what I’d really like is a record
> facility that could capture the events so I can glean how to write the
> necessary steps. Is there such a thing out there?

Not in Python.  In order to record window events, you need to "hook" the
window procedure for the window you want to monitor.  Hooking is
low-level code that requires you to inject a DLL into another processes
memory space.

Do you have the Windows Platform SDK?  It includes a tool called "spyxx"
that lets you monitor all of the window messages destined for a
particular window or process.  Very handy for this kind of thing. 
Borland had their own version called "winspector"; I see there is a
shareware tool of the same now, although I couldn't get to their web
site to check it out.

 
> Here’s the kicker though – the app I want to drive is java based. I’m
> actually wondering if this approach will work at all – for example
> despite connecting to the process with a pywinauto script I couldn’t
> seem to get to the menu structure. Beause its running in a JVM does it
> make all the UI interaction opaque from windows ?

That depends.  At the root of whatever UI framework might be piled on
top, they're all just windows, and they handle standard window
messages.  However, some applications re-invent the standard Windows UI
components using their own custom implementations.  For example, if you
bring up Microsoft Word and poke at it with "spyxx", you'll see that,
even though it looks like tool bars and status bars and a rich text
control with a standard scrollbar, all you see externally is an empty
canvas.  Word draws all of the UI components by hand.  I do not know
whether the Java UI toolkits do the same thing or not.

-- 
Tim Roberts, timr at probo.com
Providenza & Boekelheide, Inc.