Use subprocesses in simple way...

[Dara Durum]

Hi !

Py2.4, Win32.

I need to optimize a program that have a speciality: hash (MD5/SHA1)
the file contents (many files).

Now I do this in a simple python program, because (what a pity) the
FSUM utility died in a file with unicode filename... (It is unknown
error: I used alternate file name, but it not found).
And FSUM have some problematics with it's parameterizing.

So now I want choose the best solution to get hash values:
1.) Creating pyshamd5.exe with py2exe and python, and call this
program with CreateProcess and Params ?
2.) Creating pyshamd5.exe with py2exe and python, and call this
program with popen - use pipes ?
3.) Using external utility ?
4.) Other things ?

So I want to avoid the passive times like "many process starting and
ending", "many file operation", etc.
Need to use unicode filenames, so the outer program can handle these
files (alternate file names ?).
I "send" the filename to the subprocess, and it is send the hash
results (or error codes) to my main process.

Pipe good for this, but PIPE-s are sync. (synchronous) things. For
sync. things I need threads.

If I use subprocess.Popen object, it is better, because I not use
threads (I can use poll to get return code) - I can manage the
subprocesses from onethreaded code (from main thread).
(I need more process, because it speeding up my hashing. If I have one
ISO, the hashing of this file block all operations in one
thread/process. A multiprocessor code can hash more files in one
time.)

So: I need a solution that not need multithreading chaos, where I can
poll my processes.
Better, If I have persistent suprocesses, because I avoid
starting/ending "neutral gear" of subprocesses.

The working in pseudocode:

while HaveAnyFile or HaveOpenedProcesses:
    if not Processes:
       CreateProcess(MaxProcesses)
    if HaveUsableProcesses:
       fn=GetNextFile
       if fn:
           SendToSubProcess(fn)
    PollProcessesAndStoreResults()

The questions
1.)
Do you know a "method" that can send/receive datas to/from presistent
subprocesses without blocking ?
2.)
Do you know command line to just like FSUM that can compute file
hashes (MD5/SHA1), and don't have any problems with unicode alt. file
names ?
3.)
PIPE-s better than CreateProcess(params) ?

Important: the sockets are unavailable in this project. I don't use
them, because users forbid to me.

Thanks for any help:
  dd



__________________________________________________________________
Python mailing list
Levelcim:    Python at lists.fsn.hu
Webfelulet:  https://lists.fsn.hu/mailman/listinfo/python
Web oldalak: http://www.python.hu vagy http://www.python.org

Állományba menteni | Nem biztoságos képek elrejtése

Delete & Prev |  Delete & Next
Move to: