control CPU usage

[Dave Angel]

kakarukeys wrote:
> On Sep 20, 6:24 pm, Dave Angel <da... at ieee.org> wrote:
>   
>> Jiang Fung Wong wrote:
>>     
>>> Dear All,
>>>       
>>> Thank you for the information. I think I've some idea what the problem is
>>> about after seeing the replies.
>>>       
>>> More information about my system and my script
>>>       
>>> PIII 1Ghz, 512MB RAM, Windows XP SP3
>>>       
>>> The script monitors global input using PyHook,
>>> and calculates on the information collected from the events to output some
>>> numbers. Based on the numbers, the script then performs some automation
>>> using SendKeys module.
>>>       
>>> here is the memory usage:
>>> firefox.exe, 69MB, 109MB
>>> svchost.exe, 26MB, 17MB
>>> pythonw.exe, 22MB, 17MB
>>> searchindexer.exe, 16MB, 19MB
>>>       
>>> My first guess is that the script calculated for too long time after
>>> receiving an event before propagating it to the default handler, resulting
>>> the system to be non-responsive. I will try to implement the calculation
>>> part in another thread.
>>> Then the separate will have 100% CPU usage, hope the task scheduling of
>>> Windows works in my favour.
>>>       
>> (You top-posted this message, putting the whole stream out of order.  So
>> I deleted the history.)
>>
>> All my assumptions about your environment are now invalid.  You don't
>> have a CPU-bound application, you have a Windows application with event
>> loop.  Further, you're using SendKeys to generate a keystroke to the
>> other process.  So there are many things that could be affecting your
>> latency, and all my previous guesses are useless.
>>
>> Adding threads to your application will probably slow the system down
>> much more.  You need to find out what your present problem is before
>> complicating it.
>>
>> You haven't really described the problem.  You say the system is
>> unresponsive, but you made it that way by creating a global hook;  a
>> notoriously inefficient mechanism.  That global hook inserts code into
>> every process in the system, and you've got a pretty low-end environment
>> to begin with.  So what's the real problem, and how severe is it?  And
>> how will you measure improvement?  The Task manager numbers are probably
>> irrelevant.
>>
>> My first question is whether the pyHook event is calling the SendKeys
>> function directly (after your "lengthy" calculation) or whether there
>> are other events firing off  in between.  If it's all being done in the
>> one event, then measure its time, and gather some statistics (min time,
>> max time, average...).  The task manager has far too simplistic
>> visibility to be useful for this purpose.
>>
>> What else is this application doing when it's waiting for a pyHook
>> call?  Whose event loop implementation are you using?  And the program
>> you're trying to control -- is there perhaps another way in?
>>
>> DaveA
>>     
>
> Hi,
>
> Sorry I wasn't sure how to use Google groups to post a msg to the
> newsgroup, I used Gmail to write my previous reply. What you and the
> other guy have provided me isn't useless. Now I understand the non-
> responsiveness may not be caused by high CPU usage, as the OS, be it
> Windows or Linux, has a way to prioritize the tasks. This is a vital
> clue to me.
>
> By "not responsive", I mean, for some time, the mouse pointer is not
> moving smoothly, to such extent that I can't do anything with the
> mouse. It's like playing a multi-player game on a connection with a
> lot of lag. It's not caused by global hook, because it happens under
> certain condition, i.e. when fpa.ProcessEvent(word) is computing.
>
> I included my main script for your reference. Comments:
> (1) The automation method tc.Auto() is slow, but it doesn't cause any
> problem, because the user would wait for the automation to finish,
> before he continues to do something.
>
> (2) all other methods invoked are fast, except fpa.ProcessEvent(word)
> (this information is obtained from profiling). It is this method that
> causes 100% CPU usage. I'm planning to move this method to a separate
> thread, so that OnEvent(event) can finish executing, while the
> separate thread goes on to finish its calculation. Is this a good
> idea?
>
> import pyHook
> import TypingAnalyzer
> import GUI
>
> def OnEvent(event):
> 	if hasattr(event, "Key") and event.Ascii == 9 and event.Key == "Tab"
> and event.Injected == 0 and event.Alt == 0:
> 		tc.Auto()
> 		return False
> 	else:
> 		recognized = rk.ProcessEvent(event)
> 		if recognized:
> 			tc.MatchChar(recognized)
> 			paragraph = rc.ProcessEvent(recognized)
> 			if paragraph:
> 				for word in paragraph:
> 					fpa.ProcessEvent(word)
>
> 		return True
>
> hm = pyHook.HookManager()
> hm.MouseAllButtonsDown = OnEvent
> hm.KeyDown = OnEvent
> hm.HookMouse()
> hm.HookKeyboard()
>
> rk = TypingAnalyzer.ReadKey()
> rc = TypingAnalyzer.ReadChar()
> fpa =  TypingAnalyzer.Analysis()
> tc = TypingAnalyzer.Automation(fpa)
>
> if __name__ == '__main__':
> 	app = GUI.AWAApp()
> 	app.MainLoop()
>
> Thank you for your attention.
>
>   
I can't readily comment on your code, since it's entirely written to 
three imports that I have never seen.  I have to assume that 
TypingAnalyzer contains logic to automate your external program.  I have 
no idea where GUI comes from, but I have to hope it knows how to 
efficiently use the CPU, like most gui event loops.  If  you don't do 
the pyHook, is the system quite responsive?

I actually doubt if moving some of the logic to another thread is going 
to make any difference.  In Python, threading helps to overlap CPU bound 
stuff and I/O bound stuff.  But theoretically, as soon as you return to 
the event loop, your main thread is going to block, so it'll schedule 
the other thread.  The real question is whether there's communication 
going between the two processes in those various rk, rc, etc pieces.  
It's easy for such protocols to become very inefficient.

Since I can't directly help, at least I could suggest looking at XP's  
ControlPanel->System->Advanced.  That gets you to a dialog entitled 
Performance Options.

On the Advanced tab of that dialog, the first section is "processor 
scheduling".   Mine is set to "Programs" which means give extra priority 
to the GUI program that has the focus.  Presumably in your environment 
that'll be the typing code, not your python app.  Setting it to 
"Background services" would give more priority to the CPU bound of your 
python app.

But once these two apps are aware of each other, they could be doing all 
sorts of things to confuse the scheduler.

DaveA