[Web-SIG] Client-side support: what are we aiming for?

[Ian Bicking]

On Thursday, October 23, 2003, at 08:58 AM, amk at amk.ca wrote:
> What's the scope of improving client-side HTTP support?
>
> I suggest aiming for something you could write a web browser or web 
> scraper
> on top of. That means storing and returning cookies from the server, 
> writing
> them to a file, and a page cache that handles HTTP's cache expiration 
> rules.
> HTML formatting is out of scope, but a specialized parser for 
> extracting a
> list of form elements or for picking apart a table might not be.
>
> Does anyone want to produce a feature list and proposed design?

ClientCookie and ClientForm (http://wwwsearch.sourceforge.net) seem 
like a possible starting point.  I haven't used them much, but they 
seem like they resist being a framework (which is a good thing) and 
just do their one job.

I don't think you could build a browser on top of them (though that 
doesn't even apply to ClientForm, which is more of a browser 
alternative).  But if you added caching and authentication into 
ClientCookie that would probably be a reasonable basis (maybe 
authentication is already there, I don't know).  Looking at 
ClientCookie just a little more, it could even be integrated directly 
into urllib2 (it mostly matches that API already).  Really, all of this 
could go into urllib2...

--
Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org