[Python-ideas] PEP 3156 - Asynchronous IO Support Rebooted

[Markus]

Hi,

> Do you have a suggestion for a minimal interface for signal handling?
> I could imagine the following:
>
> Note that Python only receives signals in the main thread, and the
> effect may be undefined if the event loop is not running in the main
> thread, or if more than one event loop sets a handler for the same
> signal. It also can't work for signals directed to a specific thread
> (I think POSIX defines a few of these, but I don't know of any support
> for these in Python.)

Exactly - signals are a mess, threading and signals make things worse
- I'm no expert here, but I just have had experienced problems with
signal handling and threads, basically the same problems you describe.
Creating the threads after installing signal handlers (in the main
thread) works, and signals get delivered to the main thread,
installing the signal handlers (in the main thread) after creating the
threads - and the signals ended up in *some thread*.
Additionally it depended on if you'd install your signal handler with
signal() or sigaction() and flags when creating threads.

>> Supporting IOCP on windows is absolutely required, as WSAPoll is
>> broken and won't be fixed.
>> http://social.msdn.microsoft.com/Forums/hu/wsk/thread/18769abd-fca0-4d3c-9884-1a38ce27ae90
>
> Wow. Now I'm even more glad that we're planning to support IOCP.

tulip already has a workaround:
http://code.google.com/p/tulip/source/browse/tulip/unix_events.py#244

>> libuv is a wrapper around libev -adding IOCP- which adds some other
>> things besides an event loop and is developed for/used in node.js.
>
> Ah, that's helpful. I did not realize this after briefly skimming the
> libuv page. (And the github logs suggest that it may no longer be the
> case: https://github.com/joyent/libuv/commit/1282d64868b9c560c074b9c9630391f3b18ef633

Okay, they moved to libngx - nginx core library, obviously I missed this.

>> Handler - the best example for not re-using terms.
>
> ??? (Can't tell if you're sarcastic or agreeing here.)

sarcastic.

>> Fine, if you include transports, I'll pick on the transports as well ;)
>
> ??? (Similar.)

Not sarcastic.

>> Note: In libev only the "default event loop" can have timers.
>
> Interesting. This seems an odd constraint.

I'm wrong - discard. This limitation refered to watchers for child processes.

>> EventLoop
>>  - call_soon_threadsafe(callback, *args) - it would be better to have
> Not sure I understand. PEP 3156/Tulip uses a self-pipe to prevent race
> conditions when call_soon_threadsafe() is called from a signal handler
> or other thread(*) -- but I don't know if that is relevant or not.

ev_async is a self-pipe too.

> (*) http://code.google.com/p/tulip/source/browse/tulip/unix_events.py#448
> and http://code.google.com/p/tulip/source/browse/tulip/unix_events.py#576
>
>>  - getaddrinfo(host, port, family=0, type=0, proto=0, flags=0) - libev
>> does not do dns
>>  - getnameinfo(sockaddr, flags=0) - libev does not do dns
>
> Note that these exist at least in part so that an event loop
> implementation may *choose* to implement its own DNS handling (IIUC
> Twisted has this), whereas the default behavior is just to run
> socket.getaddrinfo() -- but in a separate thread because it blocks.
> (This is a useful test case for run_in_executor() too.)

I'd expect the EventLoop never to create threads on his own behalf,
it's just wrong.
If you can't provide some functionality without threads, don't provide
the functionality.

Besides, getaddrinfo() is a bad choice, as it relies on distribution
specific flags.
For example ip6 link local scope exists on every current platform, but
- when resolving an link local scope address -not domain- with
getaddrinfo, getaddrinfo will fail if no global routed ipv6 address is
available on debian/ubuntu.

>> As Transport are part of the PEP - some more:
>>
>> EventLoop
>>  * create_transport(protocol_factory, host, port, **kwargs)
>>    kwargs requires "local" - local address as tuple like
>> ('fe80::14ad:1680:54e1:6a91%eth0',0) - so you can bind when using ipv6
>> link local scope.
>>   or ('192.168.2.1',5060) - bind local port for udp
>
> Not sure I understand. What socket.connect() (or other API) call
> parameters does this correspond to? What can't expressed through the
> host and port parameters?

In case you have multiple interfaces, and multiple gateways, you need
to assign the connection to an address - so the kernel knows which
interface to use for the connection - else he'd default to "the first"
interface.
In IPv6 link-local scope you can have multiple addresses in the same
subnet fe80:: - IIRC if you want to connect somewhere, you have to
either set the scope_id of the remote, or bind the "source" address
before - I don't know how to set the scope_id in python, it's in
sockaddr_in6.

In terms of socket. it is a bind before a connect.

s = socket.socket(AF_INET6,SOCK_DGRAM,0)
s.bind(('fe80::1',0))
s.connect(('fe80::2',4712))

same for ipv4 in case you are multi homed and rely on source based routing.

>> Handler:
>> Requiring 2 handlers for every active connection r/w is highly ineffective.
>
> How so? What is the concern?

Of course you can fold the fdsets, but in case you need a seperate
handler for write, you re-create it for every write - see below.

>> Additionally, I can .stop() the handler without having to know the fd,
>> .stop() the handler, change the events the handler is looking for,
>> restart the handler with .start().
>> In your proposal, I'd create a new handler every time I want to sent
>> something, poll for readability - discard the handler when I'm done,
>> create a new one for the next sent.
>
> The questions are, does it make any difference in efficiency (when
> using Python -- the performance of the C API is hardly relevant here),
> and how often does this pattern occur.

Every time you send - you poll for write-ability, you get the
callback, you write, you got nothing left, you stop polling for
write-ability.

>> Timers:
>> ...
>> Timer.stop()
>> Timer.set(5)
>> Timer.start()
>
> Actually it's one less call using the PEP's proposed API:
>
> timer.cancel()
> timer = loop.call_later(5, callback)

My example was ill-chosen, problem for both of us - how to we know
it's 5 seconds?
timer.restart() or timer.again()

the timer could remember it's interval, else you have to store the
interval somewhere, next to the timer.

> Which of the two idioms is faster? Who knows? libev's pattern is
> probably faster in C, but that has little to bear on the cost in
> Python. My guess is that the amount of work is about the same -- the
> real cost is that you have to make some changes the heap used to keep
> track of all timers in the order in which they will trigger, and those
> changes are the same regardless of how you style the API.

Speed, nothing is fast in every circumstances, for example select is
faster than epoll for small numbers of sockets.
Let's look on usability.

>> Transports:
>> I think SSL should be a Protocol not a transport - implemented using BIO pairs.
>> If you can chain protocols, like Transport / ProtocolA / ProtocolB you can have
>> TCP / SSL / HTTP as https or  TCP / SSL / SOCKS /  HTTP as https via
>> ssl enabled socks proxy without having to much problems. Another
>> example, shaping a connection TCP / RATELIMIT / HTTP.
>
> Interesting idea. This may be up to the implementation -- not every
> implementation may have BIO wrappers available (AFAIK the stdlib
> doesn't),

Right, for ssl bios pyopenssl is required - or ctypes.

> So maybe we can visualise this as T1 <-->
> P2:T2 <--> P3:T3 <--> P4.

Yes, exactly.

>> Having SSL as a Protocol allows closing the SSL connection without
>> closing the TCP connection, re-using the TCP connection, re-using a
>> SSL session cookie during reconnect of the SSL Protocol.
>
> That seems a pretty esoteric use case (though given your background in
> honeypots maybe common for you :-). It also seems hard to get both
> sides acting correctly when you do this (but I'm certainly no SSL
> expert -- I just want it supported because half the web is
> inaccessible these days if you don't speak SSL, regardless of whether
> you do any actual verification).

Well, proper shutdown is not a SSL protocol requirement, closing the
connection hard saves some cycles, so it pays of not do it right in
large scaled deployments - such as google.
Nevertheless, doing SSL properly can help, as it allows to distinguish
from connection reset errors and proper shutdown.

> The only concern I have, really, is that the PEP currently hints that
> both protocols and transports might have pause() and resume() methods
> for flow control, where the protocol calls transport.pause() if
> protocol.data_received() is called too frequently, and the transport
> calls protocol.pause() if transport.write() has buffered more data
> than sensible. But for an object that is both a protocol and a
> transport, this would make it impossible to distinguish between
> pause() calls by its left and right neighbors. So maybe the names must
> differ. Given the tendency of transport method names to be shorter
> (e.g. write()) vs. the longer protocol method names (data_received(),
> connection_lost() etc.), perhaps it should be transport.pause() and
> protocol.pause_writing() (and similar for resume()).

Protocol.data_received rename to Protocol.io_in
Protocol.io_out - in case the transports out buffer is empty -
(instead of Protocol.next_layer_is_empty())
Protocol.pause_io_out - in case the transport wants to stop the
protocol sending more as the out buffer is crowded already
Protocol.resume_io_out - in case the transport wants to inform the
protocol the out buffer can take some more bytes again

For the Protocol limiting the amount of data received:
Transport.pause -> Transport.pause_io_in
Transport.resume -> Transport.resume_io_in

or drop the "_io" from the names, "(pause|resume_(in|out)"

>>  * reconnect() - I'd love to be able to reconnect a transport
>
> But what does that mean in general? It depends on the protocol (e.g.
> FTP, HTTP, IRC, SMTP) how much state must be restored/renegotiated
> upon a reconnect, and how much data may have to be re-sent. This seems
> a higher-level feature that transports and protocols will have to
> implement themselves.

I don't need the EventLoop to sync my state upon reconnect - just have
the Transport providing the ability.
Protocols are free to use this, but do not have to.

>> Now, in case we connect to a host by name, and have multiple addresses
>> resolved, and the first connection can not be established, there is no
>> way to 'reconnect()' - as the protocol does not yet exist.
>
> Twisted suggested something here which I haven't implemented yet but
> which seems reasonable -- using a series of short timeouts try
> connecting to the various addresses and keep the first one that
> connects successfully. If multiple addresses connect after the first
> timeout, too bad, just close the redundant sockets, little harm is
> done (though the timeouts should be tuned that this is relatively
> rare, because a server may waste significant resources on such
> redundant connects).

Fast, yes - reasonable? - no.
How would you feel if web browsers behaved like this?
domain name has to be resolved, addresses ordered according to rfc X
which says prefer ipv6 etc., try connecting linear.

>> For almost all the timeouts I mentioned - the protocol needs to take
>> care - so the protocol has to exist before the connection is
>> established in case of outbound connections.
>
> I'm not sure I follow. Can you sketch out some code to help me here?
> ISTM that e.g. the DNS, connect and handshake timeouts can be
> implemented by the machinery that tries to set up the connection
> behind the scenes, and the user's protocol won't know anything of
> these shenanigans. The code that calls create_transport() (actually
> it'll probably be renamed create_client()) will just get a Future that
> either indicates success (and then the protocol and transport are
> successfully hooked up) or an error (and then no protocol was created
> -- whether or not a transport was created is an implementation
> detail).

>From my understanding the Future does not provide any information
which connection to which host using which protocol and credentials
failed?
I'd create the Procotol when trying to create a connection, so the
Protocol is informed when the Transport fails and can take action -
retry, whatever.

>> In case aconnection is lost and reconnecting is required -
>> .reconnect() is handy, so the protocol can request reconnecting.
>
> I'd need more details of how you would like to specify this.

Transport
 * is closed by remote
 * connecting the remote failed
 * resolving the domain name failed

have inform the protocol about the failure - and if the Protocol
changes the Transports state to "reconnect", the Transport creates a
"reconnect timer of N seconds", and retries connecting then.

It is up to the protocol to login, clean state and start fresh or
login and regain old state by issuing required commands to get there.
For ftp - this would be changing the cwd.

>> As this does not work with the current Protocols callbacks I propose
>> Protocols.connection_established() therefore.
>
> How does this differ from connection_made()?

If you create the Protocol before the connection is established - you
may want to distinguish from _made() and _established().
You can not distinguish by using __init__, as it may miss the Transport arg.

> (I'm trying to follow Twisted's guidance here, they seem to have the
> longest experience doing these kinds of things. When I talked to Glyph
> IIRC he was skeptical about reconnecting in general.)

Point is - connections don't last forever, even if we want them to.
If the transport supports "reconnect" - it is still upto the protocol
to either support it or not.
If a Protocol gets disconnected and wants to reconnect -without the
Transport supporting .reconnect()- the protocol has to know it's
factory.

>>  + connection_established()
>>  + timeout_dns()
>>  + timeout_idle()
>>  + timeout_connecting()
>
> Signatures please?

 + connection_established(self, transport)
the connection is established - in your proposal it is connection_made
which I disagree with due to the lack of context in the Futures,
returns None

 + timeout_dns(self)
Resolving the domain name failed - Protocol can .reconnect() for
another try. returns None

 + timeout_idle(self)
connection was idle for some time - send a high layer keep alive or
close the connection - returns None

 + timeout_connecting(self)
connection timed out connection - Protocol can .reconnect() for
another try, returns None

>>  * data_received(data) - if it was possible to return the number of
>> bytes consumed by the protocol, and have the Transport buffer the rest
>> for the next io in call, one would avoid having to do this in every
>> Protocol on it's own - learned from experience.
>
> Twisted has a whole slew of protocol implementation subclasses that
> implement various strategies like line-buffering (including a really
> complex version where you can turn the line buffering on and off) and
> "netstrings". I am trying to limit the PEP's size by not including
> these, but I fully expect that in practice a set of useful protocol
> implementations will be created that handles common cases. I'm not
> convinced that putting this in the transport/protocol interface will
> make user code less buggy: it seems easy for the user code to miscount
> the bytes or not return a count at all in a rarely taken code branch.

Please don't drop this.

You never know how much data you'll receive, you never know how much
data you need for a message, so the Protocol needs a buffer.
Having this io in buffer in the Transports allows every Protocol to
benefit, they try to read a message from the data passed to
data_received(), if the data received is not sufficient to create a
full message, they need to buffer it and wait for more data.
So having the Protocol.data_received return the number of bytes the
Protocol could process, the Transport can do the job, saving it for
every Protocol.
Still - a protocol can have it's own buffering strategy, i.e. in case
of a incremental XML parser which does it's own buffering, and always
return len(data), so the Transport does not buffer anything.
In case the size returned by the Protocol is less than the size of the
buffer given to the protocol, the Transport erases only the consumed
bytes from the buffer, in case the len matches the size of the buffer
passed, erases the buffer.
In nonblocking IO - this buffering has to be done for every protocol,
if Transports could take care, the data_received method of the
Protocol does not need to bother.

A benefit for every protocol.

Else, every Protocol.data_received method starts with self.buffer +=
data and ends with self.buffer = self.buffer[len(consumed):]

You can even default to use a return value of None like len(data).

If you want to be fancy. you could even pass the data to the Protocol
as long as the protocol could consume data and there is data left.
This way a protocol data_received can focus on processing a single
message, if more than a single message is contained in the data - it
will get the data again - as it returned > 0, in case there is no
message in the data left, it will return 0.

This really assists when writing protocols, and as every protocol
needs it, have it in Transport.


>>  * eof_received()/connection_lost(exc) - a connection can be closed
>> clean recv()=0, unclean recv()=-1, errno, SIGPIPE when writing and in
>> case of SSL even more, it is required to distinguish.
>
> Well, this is why eof_received() exists -- to indicate a clean close.
> We should never receive SIGPIPE (Python disables this signal, so you
> always get the errno instead). According to Glyph, SSL doesn't support
> sending eof, so you have to use Content-length or a chunked encoding.
> What other conditions do you expect from SSL that wouldn't be
> distinguished by the exception instance passed to connection_lost()?

Depends on the implementation of SSL, bio/fd Transport/Protocol
SSL_ERROR_SYSCALL and unlikely SSL_ERROR_SSL.
In case of stacking TCP / SSL / http a SSL service rejecting a client
certificate for login is - to me - a connection_lost too.

>>  + nextlayer_is_empty() - called if the Transport (or underlying
>> Protocol in case of chaining) write buffer is empty
>
> That's what the pause()/resume() flow control protocol is for. You
> read the file (presumably it's a file) in e.g. 16K blocks and call
> write() for each block; if the transport can't keep up and exceeds its
> buffer space, it calls protocol.pause() (or perhaps
> protocol.pause_writing(), see discussion above).

I'd still love a callback for "we are empty".
Protocol.io_out - maybe the name changes your mind?

>> Next, what happens if a dns can not be resolved, ssl handshake (in
>> case ssl is transport) or connecting fails - in my opinion it's an
>> error the protocol is supposed to take care of
>>  + error_dns
>>  + error_ssl
>>  + error_connecting
>
> The future returned by create_transport() (aka create_client()) will
> raise the exception.

When do I get this exception - the EventLoop.run() raises?
And this exception has all information required to retry connecting?
Let's say I want to reconnect in case of dns error after 20s, the
Future raised - depending on the Exception I call_later a callback
which create_transport again?- instead of Transport.reconnect() from
the Protocol, not really easier.


MfG
Markus