[Tutor] demon / app I/F better way ?

[Alan Gauld]

> Having decided on threading the demon, and using a fifo, the queue
> module looks cool, I have one last dilema. One that quite often
causes
> me problems, that scope, namespace thing ;-)

To be honest the risks of deadlocks and race conditions using
multiple threads accessing a common queue are such that I would
tend to have the data collection thread writing to a separate
queue (out of a pool of queues) from the processing thread.
I'd then switch queues on each data change leaving the processor
reading a queue which has no risk of being overwritten by the
collector thread. I'd keep a status map of the queues such that
the collector marks them as full and the processor marks them
as empty. This way they avoid trampling on each other and if
the pool gets to the state of all full you can flag an error
and stop processing or dump it to a file before restarting
or whatever...

One advantage of using two processes linked by sockets or pipes
is that the queing is all handled for you implicitly by the
comms mechanism - the pipe/socket is inherently fifo in nature..
The danger in that case is data overflowing the available buffer
size.

The best decision in all of this depends on:
1) How often the data changes
2) how much data changes each time
3) how critical is data recovery on error conditions

If the data only changes occasionally, the changed data size is
small and you can afford to drop a data change once in a while
then any solution will do. If volumes of data are small but changes
are frequent sockets or pipes are probably best.

If changes are frequent and volumes large either fifo queues or files
is the best solution.

If changes are infrequent but the volumes large or if error
detection/correction is needed then the original files approach is
best.

Good design is always a matter of trading off one set of compromises
against the others to get a best match.

Alan G.