How To Do It Faster?!?

[Jeremy Bowers]

On Thu, 31 Mar 2005 13:38:34 +0200, andrea.gavana wrote:

> Hello NG,
> 
>       in my application, I use os.walk() to walk on a BIG directory. I
>       need
> to retrieve the files, in each sub-directory, that are owned by a
> particular user. Noting that I am on Windows (2000 or XP), this is what I
> do:

You should *try* directly retrieving the relevant information from the OS,
instead of spawning a "dir" process. I have no idea how to do that and it
will probably require the win32 extensions for Python.

After that, you're done. Odds are you'll be disk bound. In fact, you may
get no gain if Windows is optimized enough that the process you describe
below is *still* disk-bound.

Your only hope then is two things:

* Poke around in the Windows API for a function that does what you want,
and hope it can do it faster due to being in the kernel.

* Somehow work this out to be lazy so it tries to grab what the user is
looking at, instead of absolutely everything. Whether or not this will
work depends on your application. If you post more information about how
you are using this data, I can try to help you. (I've had some experience
in this domain, but what is good heavily depends on what you are doing.
For instance, if you're batch processing a whole bunch of records after
the user gave a bulk command, there's not much you can do. But if they're
looking at something in a Windows Explorer-like tree view, there's a lot
you can do to improve responsiveness, even if you can't speed up the
process overall.)