How To Do It Faster?!?

andrea_gavana at tin.it andrea_gavana at tin.it
Fri Apr 1 18:00:34 EST 2005


Hello Jeremy & NG,

>* Poke around in the Windows API for a function that does what you want,
>and hope it can do it faster due to being in the kernel.

I could try it, but I think I have to explain a little bit more my problem.

>If you post more information about how you are using this data, I can try
to help you. 

Basically, I have to scan a really BIG directory: essentially, is a UNIX
file system where all our projects resides, with thousand and thousand of
files and more than 1 TB of information. However, we are about 200-300 users
of this space. This is what I do now and I would like to improve:

1) For a particular user (1 and only 1 at a time), I would like to scan
all directories and subdirectories in order to find which FILES are owned
by this user (I am NOT interested in directory owner, only files). Noting
that I am searching only for 1 user, its disc quota is around 20-30 GB,
or something like this;
2) My application is a GUI designed with wxPython. It run on Windows, at
the moment (this is why I am asking for Windows user IDs and similar, on
Unix is much simpler);
3) While scanning the directories (using os.walk), I process the results
of my command "dir /q /-c /a-d MyDirectory" and I display this results on
a wxListCtrl (a list viewer) of wxPython in my GUI; 
4) I would not use the suggested command "dir /S" on a DOS shell because,
even if it scans recursively all directories, I am NOT able to process intermediate
results because this command never returns until it has finished to scan
ALL directories (and for 1 TB of files, it can take a LOT of time);
5) For all the files in each directory scanned, I do:
- IF a file belongs to that particular user THEN:
      Get the file name;
      Get the file size;
      Get the last modification date;
      Display the result on my wxListCtrl
- ELSE:
      Disregard the information;
- END

I get the file owner using the /Q switch of the DIR command, and I exclude
a priori the subdirectories using the /a-d switch. That because I am using
os.walk().
6) All of our users can see this big unix directory on their PC, labeled
as E:\ or F:\ or whatever. I can not anyway use UNIX command on dos (and
I can not use rsh to communicate with the unix machine and then use something
like "find . -name etc".

I hope to have been clearer this time...

I really welcome all your suggestions.

Andrea.




More information about the Python-list mailing list