Async/Concurrent HTTP Requests

[Marko Rauhamaa]

Paul Rubin <no.email at nospam.invalid>:

> Marko Rauhamaa <marko at pacujo.net> writes:
>> I have successfully done event-driven I/O using select.epoll() and
>> socket.socket().
>
> Sure, but then you end up writing a lot of low-level machinery that
> packages like twisted take care of for you.

Certainly. It would be nice if the stdlib protocol facilities were
event-driven and divorced from the low-level I/O.

Asyncio does that, of course, but the programming model feels a bit
weird.

Twisted documentation seems a bit vague on details. For example, what
should one make of this:

   def write(data):
      Write some data to the physical connection, in sequence, in a
      non-blocking fashion.

      If possible, make sure that it is all written. No data will ever
      be lost, although (obviously) the connection may be closed before
      it all gets through.

   <URL: https://twistedmatrix.com/documents/15.0.0/api/twisted.intern
   et.interfaces.ITransport.html#write>

So I'm left wondering if the call will block and if not, how is flow
control and buffering managed. The API documentation leads me to a maze
of twisted passages, all alike. From what I could gather, the write()
method is blocking and hence not suitable for serious work.

By contrast, the semantics of Python's socket.send() is crisply defined
and a pleasure to work with.


Marko