[New-bugs-announce] [issue45955] Calling read() on HTTPError may cause KeyError in tempfile

[Matt Martz]

New submission from Matt Martz <matt at sivel.net>:

HTTPError may not be fully initialized in some scenarios leading to an inconsistent interface.  This is documented in code at:

https://github.com/python/cpython/blob/55fe1ae9708d81b902b6fe8f6590e2a24b1bd4b0/Lib/urllib/error.py#L45-L50

Unfortunately the way this is implemented creates an inconsistent interface, and opaque code, without a number of inline comments explaining the behavior of HTTPError.

Additionally, the way that it currently works, will cause a KeyError to be raised from tempfile, which is rather confusing.

Instead of "partially initializing" the HTTPError object, I'd propose that when fp is None, that we provide it with something like io.BytesIO to fulfill the interface.  There may be other recommended solutions, I've not thought through this extensively yet.

I think I just prefer always calling self.__super_init but passing in something like io.BytesIO if fp is None

I'm willing to create the PR once I know which direction seems to make the most sense.

>>> from urllib.error import HTTPError
>>> from urllib.request import HTTPDigestAuthHandler, HTTPPasswordMgrWithDefaultRealm, build_opener
>>> passman = HTTPPasswordMgrWithDefaultRealm()
>>> passman.add_password(None, 'httpbin.org', 'user', 'wrong')
>>> opener = build_opener(HTTPDigestAuthHandler(passman))
>>> try:
...     opener.open('https://httpbin.org/digest-auth/auth/user/passwd')
... except HTTPError as e:
...     e.read()
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 557, in error
    result = self._call_chain(*args)
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 1238, in http_error_401
    retry = self.http_error_auth_reqed('www-authenticate',
  File ".../3.10.0/lib/python3.10/urllib/request.py", line 1111, in http_error_auth_reqed
    raise HTTPError(req.full_url, 401, "digest auth failed",


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
  File ".../3.10.0/lib/python3.10/tempfile.py", line 473, in __getattr__
    file = self.__dict__['file']
KeyError: 'file'

----------
components: Library (Lib)
messages: 407482
nosy: sivel
priority: normal
severity: normal
status: open
title: Calling read() on HTTPError may cause KeyError in tempfile
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45955>
_______________________________________