[issue21475] Support the Sitemap extension in robotparser

[Peter Wirtz]

Peter Wirtz added the comment:

Here is a patch that provides support for the Sitemap extension.

----------
keywords: +patch
Added file: http://bugs.python.org/file40791/robotparser_site_maps_v1.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21475>
_______________________________________