[issue16099] robotparser doesn't support request rate and crawl delay parameters

[Christian Heimes]

Changes by Christian Heimes <lists at cheimes.de>:


----------
keywords: +needs review
stage: test needed -> patch review

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16099>
_______________________________________