PID lockfile (was: PEP 3143: Standard daemon process library)

[Ben Finney]

(replying in ‘comp.lang.python’ for wider feedback on this issue)

On 26-Mar-2009, Francis Irving wrote:
> On Thu, Mar 26, 2009 at 12:51:06AM +1100, Ben Finney wrote:
> > The ‘python-daemon’ distribution includes a module,
> > ‘daemon.pidlockfile’. The ‘daemon.pidlockfile.PIDLockFile’ class is
> > intended to be used for this purpose.
>  
> > I am working with the developer of ‘lockfile’ to incorporate the
> > ‘PIDLockFile’ class into that library.
> 
> Ah! I didn't know about that! Can you update:
> 
> a) The example in the PEP to use it.

No, PEP 3143 is deliberately not tied to that implementation. PID file
handling is purely an interface from the point of view of that PEP.

When I said “the PIDLockFile class is intended to be used for this
purpose”, that's strictly one-way. PEP 3143 is not meant to refer to
the PIDLockFile class, only to a generic interface (context manager)
for a ‘pidfile’ object.

If the ‘pidlockfile’ module improves enough, and the ‘lockfile’
maintainer gets my changes into a release in time, I may update the
PEP to recommend that; but so far, it's outside the scope.

> b) In the documentation, give a list of different classes that might
> be appropriate to use here. Recommending that one, but pointing to
> the ones in lockfile also.

I don't have one to recommend yet:

> c) Its own documentation (maybe some doctests in the source? or some
> help?) which show how it works.
> 
> I'm still not having much luck with it though :(

This is part of why it's not recommended in the PEP; it's not really
ready. Thank you for testing it :-)

> It doesn't seem to have a constructor which sets its path, so do I
> do:
> 
> ourlockfile = daemon.pidlockfile.PIDLockFile

Presuming you mean ‘daemon.pidlockfile.PIDLockFile()’ here (that is,
get the return value of the class constructor, not the class itself).

> ourlockfile.path = '/tmp/mydaemon.pid'
> context = daemon.DaemonContext(
>     pidfile=ourlockfile,
>     stdout=logout,
>     stderr=logout
> )
> 
> If so, it doesn't work, it just exits without an error.

Can you please provide the full traceback?

> Another thing:
> 
> I've noticed there are some ^L characters in daemon.py, e.g. just
> before "class DaemonError", "def change_working_directory" etc.

Yes. That is a page break (ASCII FF), useful for printing the file or
navigating the text file by “page”. Python, like most languages,
treats them like any other whitespace in the source.

> So, I think that PIDLockFile will leave the lockfile there if the
> daemon is killed abruptly (say with "kill -9" or due to some bug).
> Certainly, all the lock classes in lockfile.py have the problem that
> it stays locked int hat circumstance.

I have my programs delete the lockfile on start-up if it's stale (i.e.
check for the existence of the file on start-up and delete it if the
referenced PID is not running). Perhaps there are better ways.

> This page (which seems pretty good anyway, and I'm sure you've
> seen!) section 6) suggests using lockf, although I believe from
> elsewhere that fcntl will do also.
> 
> http://www.enderunix.org/docs/eng/daemon.php

No, I've not seen that, but I have seen others; they tend to differ in
the details. I have looked for a more canonical reference for the
intricacies of PID file handling, but it seems to be much more ad hoc
than the definition of the daemonisation procedure.

> My impression is that the lockf is linked to the process, so if the
> process is killed the kernel will automatically free it. So my
> suggestion would be to store the pid in a pidfile, and lockf it. Not
> sure that is the exact convention used by most daemons on Debian,
> but it might be.

On Linux at least, ‘lockf’ is not defined to alter the file at all; it
doesn't cause it to be created nor removed. It is purely for a lock on
an existing file.

> It wouldn't be cross platform though. I imagine Windows code for
> this should be very different from Unix, however - making a service.

Explicitly outside the scope of PEP 3143; though it is hoped that the
described functionality will make a good basis on which to *build*
such a service, on Unix.

> Would be lovely to have something that provided one interface to
> both eventually, but probably too wild for now!

There is a skeletal PEP on the ‘python-ideas’ list for this purpose
<URL:http://mail.python.org/pipermail/python-ideas/2009-January/002606.html>.
Anyone should feel free to develop it further.

> > Thank you for your testing and feedback!
> 
> And thank you!
> 
> Your project is well worth doing, Unix daemons are so arcane, and to
> make them more Pythonic is lovely.

-- 
 \          “When we talk to God, we're praying. When God talks to us, |
  `\         we're schizophrenic.” —Jane Wagner, via Lily Tomlin, 1985 |
_o__)                                                                  |
Ben Finney <ben at benfinney.id.au>