[issue43597] robotparser should support specifying SSL context

[Tarun Chinmai Sekar]

New submission from Tarun Chinmai Sekar  <tchinmai7 at gmail.com>:

IMO this could be enhanced by adding a sslcontext parameter to read method

a sample change would it could look like
```
def read(self, sslcontext=None):
    """Reads the robots.txt URL and feeds it to the parser."""
    try:
        if sslcontext:
           f = urllib.request.urlopen(self.url, context=sslcontext)
        else:
           f = urllib.request.urlopen(self.url)
    except urllib.error.HTTPError as err:
        if err.code in (401, 403):
            self.disallow_all = True
        elif err.code >= 400 and err.code < 500:
            self.allow_all = True
    else:
        raw = f.read()
        self.parse(raw.decode("utf-8").splitlines())

```

Happy to send a PR if this proposal makes sense.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43597>
_______________________________________