OTish: using short-term TCP connections to send to multiple slaves

[Chris Angelico]

On Mon, Nov 17, 2014 at 4:21 AM, Roy Smith <roy at panix.com> wrote:
> In article <mailman.15887.1416150791.18130.python-list at python.org>,
>  Chris Angelico <rosuav at gmail.com> wrote:
>
>> UDP for anything more than your network's MTU is inefficient
>
> Why do you say it's inefficient?  Sure, the UDP datagram will get
> fragmented and re-assembled at the other end, but it's not like TCP
> would do any better.  One way or another, your data is going to be
> transmitted in packet that fit into the MTU.

Sorry, is less efficient. I was responding to the point about 64KB
being way too big for UDP, when really it's perfectly possible. It's
common to keep UDP packet sizes as low as possible to take advantage
of non-fragmentation efficiency (hence the DNS TCP switchover at 512
bytes), but I've sent some huge UDP packets around.

>> plus you'd need to roll your own acknowledgement system so you know
>> when the client got the data, at which point you're basically
>> recreating TCP.
>
> That I certainly agree with.  UDP should be used for fire-and-forget,
> where its not critical that every bit of data gets through.  A classic
> example is high-volume logging.  Once you start thinking about any kind
> of acknowledgements, you should just switch to TCP.  The alternative is
> that you will slowly end up reinventing TCP (and doing a bad job of it).

Another good use of UDP is where it's not 100% critical that *every*
update get to *every* client instantly, but a client can "catch up"
when it gets the next notification. (Since clients can be down, you'd
need some kind of catch-up mechanic anyway.) If the data involved can
fit inside a single UDP packet, this is trivially easy: just overwrite
with the new state. Otherwise, some kind of update counter works too
("hmm, I got update 5, this one's update 7, I guess I'd better catch
up"). If most clients collect most UDP updates, but occasionally they
establish a TCP connection to do the full catch-up, you can get some
handy conveniences.

>> NAT is the most common cause of breakage, but any router does have the
>> power to monitor and drop connections on any basis it likes. (I can't
>> imagine any reason a non-NAT router would want to prevent connections
>> from going idle, but it could be done.)
>
> It's common in corporate networks for routers to forcibly close idle
> connections (i.e. inject RST packets).  Often, the IT security guys
> think idle connections represent some kind of threat (and good luck
> trying to argue with them).

Yeah... but I can't imagine any reason for those connections to be a
threat. I've heard of it happening but I cannot fathom the reasoning
behind it. (You're also suggesting a much nicer approach than the
worst-case I was talking about: with RST injection, at least one end
will be immediately aware of the closure. If the router just starts
dropping packets, much harder.)

ChrisA