Multiprocessing process termination

[MRAB]

On 2014-12-31 19:33, Charles Hixson wrote:
> In order to allow multiple processes to access a database (currently
> SQLite) I want to run the process in a separate thread.  Because it will
> be accessed from multiple processes I intent to use Queues for shifting
> messages back and forth.  But none of the individuals processes will
> know when all of the others are through.  It there a way to properly
> close the database, or does it need to be done from the parent process?
>
> What I want to do is detect that a request to shutdown the Process has
> been received, and then execute a close procedure that cleans things up
> and shutdown.  Terminate is the obvious procedure to use, but that comes
> with a warning not to use it if there is an queue attached (and without
> defining attached).
> OTOH, since the queue will never be read after the process has been
> shutdown, perhaps rendering it unusable is the proper thing.
>
> Could someone comment on this step of the design?
>
> The basic idea is:
>
> class handledb(Process):
>      def __init__(self, q, otherstuff):
>          self.q  =  q
>
>     def run(self, msg):
>        while (true):
>            if  self.q.empty():
>                sleep(someamountoftime)
>            else:
>                while (not self.q.empty()):
>                      read and parse message
>                      if message says to shutdown:  self.shutdown()
>
>     def  shutdown(self):
>           close things down
>           ?? terminate ??                    <<-- should this be done
> while q is live?
>
The parent process could tell the db process to shutdown (by putting a
message in the queue) when the child processes have terminated.
Alternatively, the parent process could tell the db process how many
child processes there are and then shutdown when it has received that
many notifications from child processes that they have finished.

BTW, it's better to let the queue read block until a message arrives
than to keep polling and sleeping if it's empty.