urllib, urllib2, httplib -- Begging for consolidation?

[John J. Lee]

On 4 Jun 2002, Dave Brueck wrote:
[...]
> together. For example, what is the correct way to do a HTTP HEAD
> request that follows redirects? It's not hard, but it's silly to have
> to code it yourself if somebody already did the work for you. Well,
> httplib doesn't know how to follow an HTTP redirect, but lo and
> behold, urllib does. Unfortunately, there's no way to use it because
> in this case because urllib is for opening URLs (the GET is hardcoded
> and not easily overridable).

I suppose the standard answer applies here: it isn't there because
nobody has written it yet.  I don't think there is any disagreement about
what would be nice to have -- just about who is going to do it.

> I don't disagree that a layering or division of functionality can be
> helpful, I just disagree that the current layering/division makes
> sense.

There is no layering issue AFAICS: httplib does the basic HTTP stuff.
urllib adds some higher-level functions.  urllib2 is a essentially a
rewrite of urllib (though some of the urllib2 is implemented using
urllib), with some missing bits -- like HEAD, as you point out.

> I think it's great that opening a URL via urllib will automatically
> follow redirects, it's just a shame that you can't take advantage of
> that functionality if what you're doing falls outside the narrow
> design domain of urllib transactions. :(

We're agreed, I think.


John