[Python-checkins] [3.6] bpo-33021: Release the GIL during fstat() calls (GH-6019) (GH-6160)
Antoine Pitrou
webhook-mailer at python.org
Tue Mar 20 15:40:21 EDT 2018
https://github.com/python/cpython/commit/f3e6eadbcf4f3e0fe53f4784485b1c8464c7d282
commit: f3e6eadbcf4f3e0fe53f4784485b1c8464c7d282
branch: 3.6
author: Miss Islington (bot) <31488909+miss-islington at users.noreply.github.com>
committer: Antoine Pitrou <pitrou at free.fr>
date: 2018-03-20T20:40:18+01:00
summary:
[3.6] bpo-33021: Release the GIL during fstat() calls (GH-6019) (GH-6160)
fstat may block for long time if the file descriptor is on a
non-responsive NFS server, hanging all threads. Most fstat() calls are
handled by _Py_fstat(), releasing the GIL internally, but but
_Py_fstat_noraise() does not release the GIL, and most calls release the
GIL explicitly around it.
This patch fixes last 2 calls to _Py_fstat_no_raise(), avoiding hangs
when calling:
- mmap.mmap()
- os.urandom()
- random.seed()
(cherry picked from commit 4484f9dca9149da135bbae035f10a50d20d1cbbb)
Co-authored-by: Nir Soffer <nirsof at gmail.com>
files:
A Misc/NEWS.d/next/Library/2018-03-12-00-27-56.bpo-33021.m19B9T.rst
M Modules/mmapmodule.c
M Python/random.c
diff --git a/Misc/NEWS.d/next/Library/2018-03-12-00-27-56.bpo-33021.m19B9T.rst b/Misc/NEWS.d/next/Library/2018-03-12-00-27-56.bpo-33021.m19B9T.rst
new file mode 100644
index 000000000000..50dfafe600d8
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2018-03-12-00-27-56.bpo-33021.m19B9T.rst
@@ -0,0 +1,2 @@
+Release the GIL during fstat() calls, avoiding hang of all threads when
+calling mmap.mmap(), os.urandom(), and random.seed(). Patch by Nir Soffer.
diff --git a/Modules/mmapmodule.c b/Modules/mmapmodule.c
index 426b7cacebb5..8acb61ab98a9 100644
--- a/Modules/mmapmodule.c
+++ b/Modules/mmapmodule.c
@@ -1078,6 +1078,7 @@ static PyObject *
new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict)
{
struct _Py_stat_struct status;
+ int fstat_result = -1;
mmap_object *m_obj;
Py_ssize_t map_size;
off_t offset = 0;
@@ -1143,8 +1144,14 @@ new_mmap_object(PyTypeObject *type, PyObject *args, PyObject *kwdict)
if (fd != -1)
(void)fcntl(fd, F_FULLFSYNC);
#endif
- if (fd != -1 && _Py_fstat_noraise(fd, &status) == 0
- && S_ISREG(status.st_mode)) {
+
+ if (fd != -1) {
+ Py_BEGIN_ALLOW_THREADS
+ fstat_result = _Py_fstat_noraise(fd, &status);
+ Py_END_ALLOW_THREADS
+ }
+
+ if (fd != -1 && fstat_result == 0 && S_ISREG(status.st_mode)) {
if (map_size == 0) {
if (status.st_size == 0) {
PyErr_SetString(PyExc_ValueError,
diff --git a/Python/random.c b/Python/random.c
index e0ee153ec4a2..c62cbeb4ecb7 100644
--- a/Python/random.c
+++ b/Python/random.c
@@ -301,10 +301,15 @@ dev_urandom(char *buffer, Py_ssize_t size, int raise)
if (raise) {
struct _Py_stat_struct st;
+ int fstat_result;
if (urandom_cache.fd >= 0) {
+ Py_BEGIN_ALLOW_THREADS
+ fstat_result = _Py_fstat_noraise(urandom_cache.fd, &st);
+ Py_END_ALLOW_THREADS
+
/* Does the fd point to the same thing as before? (issue #21207) */
- if (_Py_fstat_noraise(urandom_cache.fd, &st)
+ if (fstat_result
|| st.st_dev != urandom_cache.st_dev
|| st.st_ino != urandom_cache.st_ino) {
/* Something changed: forget the cached fd (but don't close it,
More information about the Python-checkins
mailing list