performance problem of streaming data over TCP

[Changhao]

I figured out the reason. It was because of asyncore.dispatcher's 
inefficient implementation of messange sending. Instead of using its 
'push' method. I directly call the underlying 'socket.send' and got the 
problem solved. Sorry about the spam.

Thanks

Changhao wrote:
> Hi, friends,
> 
>     I am implementing a protocol on top of 'asyncore.dispatcher' to send 
> streaming multimedia data over TCP socket. However, I found that the 
> throughput of my current implementation is surprisingly low.
> 
>     Below is a snippet of my code with a note that: the packet sent over 
> the socket is of variable length, of which the first a couple bytes 
> indicates the length of the packet. I implemented a 'var_str_to_int' 
> function to get the 'length' and the offset of the payload.
> 
> ###############################################
>   def handle_read(self):
>     self.socket.setblocking(1)
>     while (True) :
> 
> 
>                                                                data = 
> self.assemble_msg()
>         if (not data) :
>             self.socket.setblocking(0)
>             return
>         (length,index) = var_str_to_int(data)
>         self._dispatch_cmd(data[index:])
>         if (not self.recv_buffer) :
>            break
>     self.socket.setblocking(0)
> 
>   def assemble_msg(self) :
>      if (self.recv_buffer) :
>           data = self.recv_buffer
>      else :
>           data = self.socket.recv(self.BUFFER_SIZE)
>           if (len(data) == 0) :
>               return None
>      length,index = var_str_to_int(data)
>      while (length +index > len(data)) :
>          data += self.socket.recv(self.BUFFER_SIZE)
>      if (length + index == len(data)) :
>          self.recv_buffer = ''
>      else :
>          self.recv_buffer = data[length+index:]
>          data = data[:length+index]
>      return data
> ###############################################
> 
>     I found it took around 7 seconds to receive a packet of 40KB sent 
> from localhost. The throughput is even not enough to support any 
> streaming multimedia file for live playback in a LAN.
> 
>     I read some threads in the group. It sounds like the problem might 
> have to do with the TCP option, setting TCP_NODELAY probably could solve 
> the problem. But I tried the following on both a Linux and a windows XP 
> machine. Neither of them worked:
> 
> ##############################################
> self.socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
> ##############################################
> 
>     Any comments or suggests is highly appreciated.
> 
>     Thanks,
>     Changhao