[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

miss-islington report at bugs.python.org
Sun Jun 16 03:10:09 EDT 2019


miss-islington <mariatta.wijaya+miss-islington at gmail.com> added the comment:


New changeset 45d6547acfb9ae1639adbe03dd14f38cd0642ca2 by Miss Islington (bot) in branch '3.7':
bpo-35922: Fix RobotFileParser when robots.txt has no relevant crawl delay or request rate (GH-11791)
https://github.com/python/cpython/commit/45d6547acfb9ae1639adbe03dd14f38cd0642ca2


----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35922>
_______________________________________


More information about the Python-bugs-list mailing list