subeen wrote: > can use urllib2 module and/or beautiful soup for developing crawler Not if you care about a) speed and/or b) memory efficiency. http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/ Stefan