[issue30441] os.environ raises RuntimeError: dictionary changed size during iteration

[Osvaldo Santana Neto]

New submission from Osvaldo Santana Neto:

We're facing ocasional RuntimeError exceptions in a multithreaded application when one of the threads creates new entries to the environment (os.environ).

I'm not so sure if the attached patch fixes this issue the right way, so, feel free to propose another approach.

Traceback Sample of the issue in our production environment:


RuntimeError: dictionary changed size during iteration
  File "python3.5/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)


  [ internal code ]


  File "newrelic/api/background_task.py", line 103, in wrapper
    return wrapped(*args, **kwargs)
  File "python3.5/contextlib.py", line 30, in inner
    return func(*args, **kwds)
  File "python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "raven/base.py", line 851, in make_decorator
    yield
  File "python3.5/contextlib.py", line 30, in inner
    return func(*args, **kwds)
  

  [ internal code ]


  File "retry/api.py", line 74, in retry_decorator
    logger)
  File "retry/api.py", line 33, in __retry_internal
    return f()


  [ internal code ]


  File "requests/sessions.py", line 531, in get
    return self.request('GET', url, **kwargs)
  File "requests/sessions.py", line 509, in request
    prep.url, proxies, stream, verify, cert
  File "requests/sessions.py", line 686, in merge_environment_settings
    env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
  File "requests/utils.py", line 696, in get_environ_proxies
    return getproxies()
  File "urllib/request.py", line 2393, in getproxies_environment
    for name, value in os.environ.items():
  File "_collections_abc.py", line 676, in __iter__
    for key in self._mapping:
  File "python3.5/os.py", line 702, in __iter__
    for key in self._data:

----------
components: Library (Lib)
files: fix_os_environ_iteration_issue.diff
keywords: patch
messages: 294250
nosy: osantana
priority: normal
severity: normal
status: open
title: os.environ raises RuntimeError: dictionary changed size during iteration
type: behavior
versions: Python 3.5, Python 3.6
Added file: http://bugs.python.org/file46889/fix_os_environ_iteration_issue.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue30441>
_______________________________________