Fundamental problem with urllib...

[Jeremy Hylton]

"A.M. Kuchling" <akuchlin at ute.mems-exchange.org> wrote in message news:<slrnacb5ko.4i6.akuchlin at ute.mems-exchange.org>...
> In article <yNUw8.74422$T%5.18813 at atlpnn01.usenetserver.com>, 
> 	Steve Holden wrote:
> > Since urllib knows nothing of cookies, you will need to integrate some sort
> > of a cookie jar into the library, with a new API  for the clients to
> > retrieve and store the cookies.
> 
> This is worthwhile, but I don't think it belongs in urllib.  It
> belongs in a module or package of its own that provides general
> Web-browser features such as cookies, remembering authentication
> usernames and passwords, and a cache.  This package could then be used
> for implementing HTML-scraping scripts, spiders, or a Web browser.

urllib2 provides a more flexible framework for implementing
URL-loading programs, like a spider.  I think it would be helpful to
have the features you mention integrated into urllib2.

I'm not sure what the difference between an HTTP client, like urllib
or urllib2, and a Web-browser is.  Other than urllib's monolithic
design, why wouldn't you want these sorts of features in the module?

Jeremy