asynchat sends data on async_chat.push and .push_with_producer

[Josiah Carlson]

Ludvig,

In a substantial way, I agree with you.  Calling initiate_send()
within push or push_with_producer is arguably a misfeature (which you
have argued).

In a pure world, the only writing that is done would be within the
handle_send() callbacks within the select loop.  Then again, in a
perfect world, calling readable() and writable() would have no strange
side affects (as your example below has), and all push*() calls would
be made within the handle_*() methods.

We do not live in a pure world, Python isn't pure (practicality beats
purity), and by attempting to send some data each time a .push*()
method is called, there are measurable increases in transfer rates.

In the particular case you are looking at (and complaining about ;) ),
if you want to bypass the initiate_send() call, you can dig into the
particular implementation of asynchat you are using (the internals may
change in 2.6 and 3.x versus 2.5 and previous), and append your output
to the outgoing queue.  You could even abstract out the push*() calls
for a non-auto-sending version (easy), write your own initiate_send()
method that checks the stack to verify that it's being called from
handle_send() (also easy), or any one of many other work-arounds.

Yes, it would be convenient to not have push*() actually send data
when called in some cases, but in others, the increase in data
transfer rates and/or reduction in latency is substantial.

 - Josiah

ludvig.ericson at gmail.com wrote:
> Hello,
>
> My question concerns asynchat in particular. With the following half-
> pseudo code in mind:
>
>     class Example(asynchat.async_chat):
>         def readable(self):
>             if foo:
>                 self.push_with_producer(ProducerA())
>             return asynchat.async_chat.readable(self)
>
> Now, asyncore will call the readable function just before a select(),
> and it is meant to determine whether or not to include that asyncore
> dispatcher in the select map for reading.
>
> The problem with this code is that it has the unexpected side-effect
> of _immediately_ trying to send, disregarding if the async_chat object
> is indeed writable or not.
>
> The asynchat.push_with_producer (and .push as well)
> call .initiate_send(), which in turn calls .send if there's data
> buffered. While this might seem logical, it isn't at all.
>
> Insinuate that when Example.readable is called, the socket has already
> been closed. There are two possible scenarios where it could be
> closed. a) The remote endpoint closed the connection, and b) the
> producer ProducerA somehow closed the connection (my case).
>
> Obviously, calling send on a socket that has been closed will result
> in an error - EBADF, "Bad file descriptor".
>
> So, my question is: Why does asynchat.push* call self.initiate_send?
> Nothing in the name "push" suggests that it'll transmit immediately,
> disregarding potential "closedness". Removing the two calls
> to .initiate_send() in the two push functions would still mean data is
> sent, but only when data can be sent - which is, IMO, what should be
> done.
>
> Thankful for insights,
> Ludvig.