[issue16099] robotparser doesn't support request rate and crawl delay parameters

[Berker Peksag]

Berker Peksag added the comment:

I left a few comments on Rietveld.

----------
nosy: +berker.peksag
versions: +Python 3.5 -Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16099>
_______________________________________