2 daemons write to a single file /w python file IO

[Evan Klitzke]

On Tue, 2007-09-11 at 21:17 -0700, Andrey wrote:
> HI
> 
> i have a newbie question about the file() function.
> I have 2 daemons running on my linux box.
> 
> 1 will record the IDs to a file - logs.txt
> other 1 will open this file, read the IDs, and then "Clean up the 
> file" -logs.txt
> 
> Since these 2 daemons will run every 2-5mins, I think this will crash, isn't 
> it? When both daemons try to write to the file at the same time.

They will not crash. If you're writing a small amount of data using
os.write (e.g. a line or two at a time), the writes will probably end up
being atomic, and you don't have to worry about anything. This isn't
guaranteed of course, but if you're not writing out large buffers you
probably don't need to worry about it at all. The worst case scenario is
that the writes will be interleaved (e.g. daemon 1 writes "foo", daemon
2 writes "bar", and what ends up in the file is something like
"fbaroo"), there is not danger of crashing. The regular write method of
file objects is buffered, so I'd imagine that is more likely to have
this problem.

> I am wondering if this won't crash, OR if there is some simple high-level 
> functions can lock the file while writing...

There is a standard syscall in POSIX system that does real file locking
-- it's called flock. You can access it using the fcntl module as
fcntl.flock. The man page for flock has more details.

Basically what you need to do is call fcntl.flock before writing to the
file in each of your processes. The process will block until the lock is
released. On Linux the lock will be advisory only, so if you don't use
flock everywhere you can still have multiple processes with the file
descriptor open and writable at once.

> I also wonder if one side locked the file, what happens if the other side 
> try to open this locked file? raise error? so i also need to write a loop to 
> wait for the file to release locking?

The flock call will block if the file is already locked.

-- 
Evan Klitzke <evan at yelp.com>