Running a python program during idle time only

[Mike Meyer]

"los" <carlos80 at gmail.com> writes:

> Thanks for all the replies.
> 
> I did try using nice under windows.  I created a java program that
> would just loop and print numbers on the screen.  Even when I ran that
> simple program with nice, (lets call it program A) as soon as I started
> the program the cpu went all the way to 100% usage.  Then when I ran
> another program that did the same thing (lets call it program B),
> program A halted to let B finish, then it started again.  Nevertheless
> it still hogged all the cpu while I was using the computer.

Actually, CPU usage going to 100% when you run a cpu-intensive program
under nice is what you expect - even want. The idea is that the nice'd
prog soaks up all the "unused" cpu on the system. You don't say whether
program B was niced or not, but from the description, it wasn't. And you
got the behavior I thought you wanted: the nice'd program quit using
any CPU at all while the normal program was running.

At least, that's the behavior I want from my backgrounded tasks.

> For my indexing program I just wrote a simple python program and called
> on the python os.walk() method to iterate through the drive and then it
> connects to a database to store some information.  Then I wrote a
> simple interface to connect to the database to search for files using
> visual basic.  Once everything is indexed it works fine, but it would
> be nice to have the program looping through and indexing the files all
> the time to account to file updates, deletes, and relocation, but
> without hurting the performance when I'm using the computer.

As has been pointed out, your daemon is doing disk i/o, which nice won't
mediate properly.

> So really what I am looking for is some way to have the program only
> start indexing and crawling through the hd after 5 minutes of no user
> interaction with the computer.

Why 5 minutes? What if you've gone to get a cup of coffee while something
that takes more than five minutes is completing? 

I'm sure there are tools for doing this kind of thing on windows. That's
pretty much how screen savers work. I haven't written a screen saver
for windows, though - so I have no idea what you're looking for.

Note that the screen savers I have written still lowered their priority
to avoid interfering with any long-running tasks you may be waiting on.
At least for CPU usage.

> I'm going to take a look at this CPU load possibility.  But I'm afraid
> that this will work similarly to "nice" in which case it will let the
> program kick in when the CPU isn't being used heavily, but I might
> still be using the computer.  

I think you're right.

On a completely different topic, this looks like the wrong way to solve
the problem. You want to update a search engine based on changes to the
underlying file system. The right way to do this isn't to just keep
rescanning the file system, it's to arrange things so that your scanner
gets notified of any changes made to the file system. I did something like 
this for my web site search engine, but that's hooked into the SCM that's
used for propogating changes to the web site. I know someone is working
on patches to the FreeBSD kernel to make this kind of thing work. It would
seem that some of the "backup" facilities that worked by keeping a mirror
of the disk on separate media would have to have used such hooks, but maybe
not.

I'm not sure this is possible. If it is, it's almost certainly deep magic.

        <mike
-- 
Mike Meyer <mwm at mired.org>			http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.