[New-bugs-announce] [issue35334] urllib3 fails with type error exception, when cannot reach PyPI - urllib3.util.retry

[Arno-Can Uestuensoez]

New submission from Arno-Can Uestuensoez <acue.opensource at gmail.com>:

Hi,
I am using the installation script *adafruit-pitft.sh* which tries to apply *urllib3* which itself tries to install nested dependencies,
here *evdev* from PyPI.

When the installation is offline - e.g. by internal mirror of debian/raspbian - PyPI is not accessible. The code fails than by an type error
exception:

> adafruit-pitft.sh # or pitft.sh 
> https://github.com/adafruit/Raspberry-Pi-Installer-Scripts/blob/master/adafruit-pitft.sh
>
> ...
> 
>   _stacktrace=sys.exc_info()[2])
>  File "/usr/share/python-wheels/urllib3-1.13.1-py2.py3 none-any.whl/urllib3/util/retry.py", line 228, in increment
>
>    total -= 1
>
> TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'

For the current distribution(based on debian-9.6.0/stretch):

>  File "/usr/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 315, in increment
>
>    total -= 1
>
> TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'

See also: https://stackoverflow.com/questions/37495375/python-pip-install-throws-typeerror-unsupported-operand-types-for-retry


The following - dirty *:) - patch enables a sounding error trace:

    # File: retry.py - in *def increment(self, ..* about line 315
    # original: total = self.total
    
    # patch: quick-and-dirty-fix
    # START:
    if isinstance(self.total, Retry):
        self.total = self.total.total
    
    if type(self.total) is not int:
        self.total = 2 # default is 10
    # END:
    
    # continue with original:
    total = self.total

    if total is not None:
        total -= 1
    
    connect = self.connect
    read = self.read
    redirect = self.redirect
    cause = 'unknown'
    status = None
    redirect_location = None
    
    if error and self._is_connection_error(error):
        # Connect retry?
        if connect is False:
            raise six.reraise(type(error), error, _stacktrace)
        elif connect is not None:
            connect -= 1


The sounding output with the temporary patch is(twice...):

> Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at/
>
>  Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 
'ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at/
>
>  Could not find a version that satisfies the requirement evdev (from versions: )
>
> No matching distribution found for evdev
>
> WARNING : Pip failed to install software!

In this case I did not thoroughly analysed the actual error source, thus did a quick and dirty patch only.

My proposal is to add the inplace operator to the class *Reply* with the default assignment in case of an type error and display an error message.
This is still not really the correct solution, but resolves a lot of confusion by required interpretation of the exception.

Arno-Can Uestuensoez

----------
components: Library (Lib)
messages: 330574
nosy: Quentin.Pradet, acue, deivid, martin.panter
priority: normal
severity: normal
status: open
title: urllib3 fails with type error exception, when cannot reach PyPI - urllib3.util.retry
type: crash
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35334>
_______________________________________