socket bug or not?

[Gabriel Rossetti]

Hello everyone,

I found what was not working in my code (see Twisted mailing list topics 
"self.socket.accept() in doRead() in tcp.py has (11, 'Resource 
temporarily unavailable') error", "Twisted, PushProducer, Words & big 
msgs, any limit?" and Python mailing list topic "socket.send : (11, 
'Resource temporarily unavailable')"). I was just wondering if this was 
a bug or not, so here it is, I wrote aserver using twisted words' 
xmlstream class. When a client connects, it sends "<stream>" and the 
server also sends it, so a bi-directional xmlstream communication is 
established. This works fine, I at some point wrote a webservice to send 
msgs to my system, not using any twisted code, that sends a message to 
my server in this format : "<stream><message>.....</message></stream>". 
This worked fine as long as the messages were small, but if it send 
larger msgs (I tried +128k msgs) the server only received part of the 
msg and then the parser died because of this, so my msg appeared to be 
dropped. I discovered that it was because my webservice did not read the 
"<stream>" element sent by the server when it connected (since I was not 
interested in having a bidirectional xml stream communication with it), 
when I added the code to read it, everything worked as expected. I 
tested the webservice to without the server to see if it had a problem, 
using netcat, and it does, so I wonder if there is a bug in the socket 
code or not. Here is the test code :


###################### Listing 1 Start ######################
import socket, time
STREAM_START = "<stream>"
STREAM_END = "</stream>"

def sendTestMessage(host, port):
    msg = "<message>" + ('a' * 175177) + "</message>"
    burstSize = 4096
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect((host, port))
    sock.send(STREAM_START)
    time.sleep(5)
    while msg:
        sent = sock.send(msg[:burstSize])
        print "Sending %d bytes..." % sent
        if sent == 0:
            raise RuntimeError, "socket connection broken"
        msg = msg[burstSize:]
    sock.send(STREAM_END)
    sock.close()
###################### Listing 1 End ######################

To test it:

 1) open a python interpretor, copy-past the code above
 2) open another terminal and run netcat using the following command : 
nc -v -l -p 4444
 3) call the above function using : sendTestMessage("localhost", 4444), 
it will wait 5 seconds after having sent the <stream>, for the first 
test, just wait.

The message will be completely send and received by the netcat "server", 
now let's test the case I have, where the server sends a <stream>, to do 
that, repeat the above steps (you have to re-run netcat as it exits when 
the client disconnects), except in step 3 instead of just waiting, type 
<stream> and press enter in the netcat terminal after having received 
the <stream> element from the sending code. This time you will see that 
the message is incomplete.

If you send a smaller message, like by replacing the following line :

msg = "<message>" + ('a' * 175177) + "</message>"

with :

msg = "<message>" + ('a' * (175177/4)) + "</message>"

it works in both cases.

Now test the "fixed" code :

###################### Listing 2 Start ######################
import socket
STREAM_START = "<stream>"
STREAM_END = "</stream>"

def sendTestMessage(host, port):
    msg = "<message>" + ('a' * 175177) + "</message>"
    burstSize = 4096
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect((host, port))
    sock.send(STREAM_START)
    sock.recv(len(STREAM_START)+1)
    while msg:
        sent = sock.send(msg[:burstSize])
        print "Sending %d bytes..." % sent
        if sent == 0:
            raise RuntimeError, "socket connection broken"
        msg = msg[burstSize:]
    sock.send(STREAM_END)
    sock.close()
###################### Listing 2 End ######################

This time the code reads for the <stream> element after sending the 
<stream> element so it works, just try the steps described above, the 
ones where you have to type <stream> and press enter in the netcat 
terminal and this time it works.

Is this a bug in the sockets code, or is this normal? If it's normal I 
think it should be mentioned somewhere. Oh, by the way, I use linux and 
this code was tested only on linux, I don't know if the problem also 
occurs on other platforms.

Gabriel