OTish: using short-term TCP connections to send to multiple slaves

[Alain Ketterlin]

jkn <jkn_gg at nicorp.f9.co.uk> writes:

> I have a use case of a single 'master' machine which will need to 
> periodically 'push' data to a variety of 'slave' devices on a small local 
> subnet, over Ethernet. We are talking perhaps a dozen devices in all with 
> comms occurring perhaps once very few seconds, to much less often - once per 
> half an hour, or less. There is probably an upper bound of 64KB or so of 
> data that is likely to be sent on each occasion.

OK, no big requirements, but 64K is still way too much to consider UDP.

> Previous similar systems have attempted to do this by maintaining multiple 
> long-term TCP connections from the master to all the slave devices. The 
> Master is the server and the slaves periodically check the master to see 
> what has changed. Although this ... works ..., we have had trouble 
> maintaining the connection, for reasons ... I am not yet fully aware
> of.

This doesn't make much sense to me. On a packet switching network
"maintaining the connexion" simply means keeping both ends alive. There
is nothing "in the middle" that can break the connection (unless you
use, e.g., ssh tunneling, but then it's a configuration problem, or NAT,
but I doubt it is your case on an LAN).

> We are now considering an alternative approach where the master maintains a 
> list of slave devices and acts as a client. Each device is a server from the 
> point of view of data transfer. We would then use a short-lived TCP 
> connection when data is available; the Master would connect to each slave 
> which needed the data, send it, and close the connection.

Yes but a TCP server is slightly more complex: it has to "accept()"
connexions, which means it blocks waiting for something to come in. Or
it has to periodically poll its passive socket, but then you have a
problem with timeouts. Your describing the slaves as "devices" makes me
think they are simple machines, maybe embedded micro-controllers. If it
is the case, you may not want to put the burden on slaves.

(BTW why not have the slaves periodically connect -- as clients -- to
the master? No need to keep the connection open, they can disconnect and
reconnect later. All you need is a way for the master to identify slaves
across several connections, but an IP address should be enough, no?)

> I should also add that we desire our system to be 'robust' in the face of 
> situations such as cable unplugging, device power cycles, etc.

Then avoid connected protocols like TCP. Maybe you should consider SCTP,
which is a message-oriented, reliable protocol. There is a pysctp module
on pypi. (Haven't tried it, I've just made a quick search on SCTP in
Python.)

Or roll your own protocol on top of UDP, but that's another story.

[...]

-- Alain.