[Tutor] socket communications and threading

[richard kappler]

I'm having difficulty wrapping my arms around sockets and threading, not so
much from a 10,000 foot/ network perspective, but from a low level
perspective.

In our production environment we have three machines that generate data and
forward it over tcp to a computer that stores the data, parses it and sends
it on to another computer over tcp for analytics.

In our test environment we have simulated this by building three vm's. VM1
has a python script that sends raw data over tcp to VM2 which parses the
data and sends it over tcp to VM3 upon which we are developing our
analytics apps.

Note that VM1 script reads from a .log file which is the actual output from
three real machines, differentiated by a unique deviceID, and each line or
'event' in the .log file is read and sent over tcp with a 0.5 second delay
between each read.

VM2 has a python script that receives through a socket the raw data over
tcp, parses it and sends it on through a socket over tcp to VM3.

My lack of confusion arises for a couple reasons.

1. The data from the three different machines each gets it's own thread in
production, so that would have to happen on 'VM2' as the 'VM1' are actually
just microcontrollers out  in production. From a socket and threading
perspective, which would be considered the client  and which the server,
VM1 (the sender) or VM2 (the receiver)?

2. The process has worked mediocre at best thus far. When I developed the
two python scripts (tunnelSim to send over socket and parser to rx and tx
over socket) I started by just reading and writing to files so I could
concentrate on the parsing bit. Once that was done and worked very well I
added in sockets for data flow and commented out the read from and to files
bits, and everything seemed to work fine, VM1 sent a number of 'lines', VM2
received the same number of 'lines', parsed them and, seemed to send them
on. Some days analytics (VM3) got them all, some days it did not. Not sure
where to look, and any thoughts on troubleshooting this would be helpful,
but the main point of the entire email is question 1, threading.

regards, Richard