[Python-checkins] r71036 - in python/trunk: Doc/library/multiprocessing.rst Misc/NEWS Modules/_multiprocessing/connection.h Modules/_multiprocessing/pipe_connection.c

[jesse.noller]

Author: jesse.noller
Date: Thu Apr  2 06:22:09 2009
New Revision: 71036

Log:
Issue 3551: Raise ValueError if the size causes ERROR_NO_SYSTEM_RESOURCES

Modified:
   python/trunk/Doc/library/multiprocessing.rst
   python/trunk/Misc/NEWS
   python/trunk/Modules/_multiprocessing/connection.h
   python/trunk/Modules/_multiprocessing/pipe_connection.c

Modified: python/trunk/Doc/library/multiprocessing.rst
==============================================================================
--- python/trunk/Doc/library/multiprocessing.rst	(original)
+++ python/trunk/Doc/library/multiprocessing.rst	Thu Apr  2 06:22:09 2009
@@ -710,7 +710,8 @@
       Send an object to the other end of the connection which should be read
       using :meth:`recv`.
 
-      The object must be picklable.
+      The object must be picklable.  Very large pickles (approximately 32 MB+,
+      though it depends on the OS) may raise a ValueError exception.
 
    .. method:: recv()
 
@@ -742,7 +743,9 @@
       complete message.
 
       If *offset* is given then data is read from that position in *buffer*.  If
-      *size* is given then that many bytes will be read from buffer.
+      *size* is given then that many bytes will be read from buffer.  Very large
+      buffers (approximately 32 MB+, though it depends on the OS) may raise a
+      ValueError exception
 
    .. method:: recv_bytes([maxlength])
 

Modified: python/trunk/Misc/NEWS
==============================================================================
--- python/trunk/Misc/NEWS	(original)
+++ python/trunk/Misc/NEWS	Thu Apr  2 06:22:09 2009
@@ -202,6 +202,10 @@
 Library
 -------
 
+- Issue 3551: Patch multiprocessing to raise a proper exception if the size of the
+  object when writefile is called causes a ERROR_NO_SYSTEM_RESOURCES. Added docs
+  to note the limitation
+
 - unittest.assertNotEqual() now uses the inequality operator (!=) instead 
   of the equality operator.
   

Modified: python/trunk/Modules/_multiprocessing/connection.h
==============================================================================
--- python/trunk/Modules/_multiprocessing/connection.h	(original)
+++ python/trunk/Modules/_multiprocessing/connection.h	Thu Apr  2 06:22:09 2009
@@ -131,8 +131,12 @@
 
 	res = conn_send_string(self, buffer + offset, size);
 
-	if (res < 0)
-		return mp_SetError(PyExc_IOError, res);
+	if (res < 0) {
+		if (PyErr_Occurred())
+			return NULL;
+		else
+			return mp_SetError(PyExc_IOError, res);
+	}
 
 	Py_RETURN_NONE;
 }

Modified: python/trunk/Modules/_multiprocessing/pipe_connection.c
==============================================================================
--- python/trunk/Modules/_multiprocessing/pipe_connection.c	(original)
+++ python/trunk/Modules/_multiprocessing/pipe_connection.c	Thu Apr  2 06:22:09 2009
@@ -23,6 +23,12 @@
 	Py_BEGIN_ALLOW_THREADS
 	ret = WriteFile(conn->handle, string, length, &amount_written, NULL);
 	Py_END_ALLOW_THREADS
+
+	if (ret == 0 && GetLastError() == ERROR_NO_SYSTEM_RESOURCES) {
+		PyErr_Format(PyExc_ValueError, "Cannnot send %" PY_FORMAT_SIZE_T "d bytes over connection", length);
+		return MP_STANDARD_ERROR;
+	}
+
 	return ret ? MP_SUCCESS : MP_STANDARD_ERROR;
 }