[issue36207] robotsparser deny all with some rules
Nicolas
report at bugs.python.org
Sun Oct 25 18:55:38 EDT 2020
Nicolas <toulouse at meridigital.com> added the comment:
Seems like we have the same issue with http://meridigital.com/robots.txt
----------
nosy: +nico702 -matthieuhemea
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36207>
_______________________________________
More information about the Python-bugs-list
mailing list