[Python-checkins] gh-61460: Add a comment describing the multiprocessing.connection protocol (gh-99623)

[gpshead]

https://github.com/python/cpython/commit/abf5b6ff43c5e238e2d577c95ed27bc8ff01afd5
commit: abf5b6ff43c5e238e2d577c95ed27bc8ff01afd5
branch: main
author: Gregory P. Smith <greg at krypto.org>
committer: gpshead <greg at krypto.org>
date: 2022-11-20T10:20:04-08:00
summary:

gh-61460: Add a comment describing the multiprocessing.connection protocol (gh-99623)

Describe the multiprocessing connection protocol.

It isn't a good protocol, but it is what it is.  This way we can more
easily reason about making changes to it in a backwards compatible way.

files:
M Lib/multiprocessing/connection.py

diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py
index b08144f7a1a1..1a8822b9db01 100644
--- a/Lib/multiprocessing/connection.py
+++ b/Lib/multiprocessing/connection.py
@@ -728,6 +728,74 @@ def PipeClient(address):
 WELCOME = b'#WELCOME#'
 FAILURE = b'#FAILURE#'
 
+# multiprocessing.connection Authentication Handshake Protocol Description
+# (as documented for reference after reading the existing code)
+# =============================================================================
+#
+# On Windows: native pipes with "overlapped IO" are used to send the bytes,
+# instead of the length prefix SIZE scheme described below. (ie: the OS deals
+# with message sizes for us)
+#
+# Protocol error behaviors:
+#
+# On POSIX, any failure to receive the length prefix into SIZE, for SIZE greater
+# than the requested maxsize to receive, or receiving fewer than SIZE bytes
+# results in the connection being closed and auth to fail.
+#
+# On Windows, receiving too few bytes is never a low level _recv_bytes read
+# error, receiving too many will trigger an error only if receive maxsize
+# value was larger than 128 OR the if the data arrived in smaller pieces.
+#
+#      Serving side                           Client side
+#     ------------------------------  ---------------------------------------
+# 0.                                  Open a connection on the pipe.
+# 1.  Accept connection.
+# 2.  New random 20 bytes -> MESSAGE
+# 3.  send 4 byte length (net order)
+#     prefix followed by:
+#       b'#CHALLENGE#' + MESSAGE
+# 4.                                  Receive 4 bytes, parse as network byte
+#                                     order integer. If it is -1, receive an
+#                                     additional 8 bytes, parse that as network
+#                                     byte order. The result is the length of
+#                                     the data that follows -> SIZE.
+# 5.                                  Receive min(SIZE, 256) bytes -> M1
+# 6.                                  Assert that M1 starts with:
+#                                       b'#CHALLENGE#'
+# 7.                                  Strip that prefix from M1 into -> M2
+# 8.                                  Compute HMAC-MD5 of AUTHKEY, M2 -> C_DIGEST
+# 9.                                  Send 4 byte length prefix (net order)
+#                                     followed by C_DIGEST bytes.
+# 10. Compute HMAC-MD5 of AUTHKEY,
+#     MESSAGE into -> M_DIGEST.
+# 11. Receive 4 or 4+8 byte length
+#     prefix (#4 dance) -> SIZE.
+# 12. Receive min(SIZE, 256) -> C_D.
+# 13. Compare M_DIGEST == C_D:
+# 14a: Match? Send length prefix &
+#       b'#WELCOME#'
+#    <- RETURN
+# 14b: Mismatch? Send len prefix &
+#       b'#FAILURE#'
+#    <- CLOSE & AuthenticationError
+# 15.                                 Receive 4 or 4+8 byte length prefix (net
+#                                     order) again as in #4 into -> SIZE.
+# 16.                                 Receive min(SIZE, 256) bytes -> M3.
+# 17.                                 Compare M3 == b'#WELCOME#':
+# 17a.                                Match? <- RETURN
+# 17b.                                Mismatch? <- CLOSE & AuthenticationError
+#
+# If this RETURNed, the connection remains open: it has been authenticated.
+#
+# Length prefixes are used consistently even though every step so far has
+# always been a singular specific fixed length.  This may help us evolve
+# the protocol in the future without breaking backwards compatibility.
+#
+# Similarly the initial challenge message from the serving side has always
+# been 20 bytes, but clients can accept a 100+ so using the length of the
+# opening challenge message as an indicator of protocol version may work.
+
+
 def deliver_challenge(connection, authkey):
     import hmac
     if not isinstance(authkey, bytes):