2 daemons write to a single file /w python file IO

[Steven W. Orr]

On Tuesday, Sep 11th 2007 at 21:17 -0700, quoth Andrey:


=>i have a newbie question about the file() function.
=>I have 2 daemons running on my linux box.
=>
=>1 will record the IDs to a file - logs.txt
=>other 1 will open this file, read the IDs, and then "Clean up the 
=>file" -logs.txt
=>
=>Since these 2 daemons will run every 2-5mins, I think this will crash, isn't 
=>it? When both daemons try to write to the file at the same time.
=>
=>I am wondering if this won't crash, OR if there is some simple high-level 
=>functions can lock the file while writing...
=>I also wonder if one side locked the file, what happens if the other side 
=>try to open this locked file? raise error? so i also need to write a loop to 
=>wait for the file to release locking?
=>
=>seems a basic easy thing but i just cannot find an simple answer.
=>I always advoid this issue by putting them in mysql (well, fast and hassle 
=>free for locking)

This is more of a unix question as opposed to a python question, but I'll 
'splain it anyways :-)

If two processes each have a channel opened to the same file then it is 
very possible for the writes to collide. One way to mitigate the damage 
would be to make sure that the two processes open the file for append 
access instead of simple write access. This will ensure that the if 
ProcessA writes data that ProcessB's next write won't overwrite the data 
that A just wrote. But! It still won't work! You will still get 
interleaving of data in your output file. They will all get out there, but 
the data will on occasion look garbled because the interleaving won't look 
like it was partitioned as the different processes intended.

So what's the solution? It turns out that data that is written to a pipe 
(as opposed to data that is written to a file) is guaranteed to maintain 
its partitioned integrity.

P1 >> fn
P2 >> fn

If you are the guy who starts P1 and P2 from a parent process(PP) then you 
could structure your pipes so you end up with 

              PP > fn
             / \
            /   \
           P1   P2

and just make PP be a multiplexor.  :-)

-- 
Time flies like the wind. Fruit flies like a banana. Stranger things have  .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net