[Web-SIG] WSGI in standard library

[Ian Bicking]

Bill Janssen wrote:
>>   I think John Lee has done some work on this?  Beats me.  I've never 
>>felt any need for CSS parsing personally.
>>
> 
> 
> If you are doing any work with the Web (spidering, for instance), and
> need to do rendering of the Web pages (say, for pop-out prism, or
> building an ebook from it), a CSS parser is pretty much necessary.  I
> tend to think that you really can't understand the Web
> programmatically without a CSS parser.
> 
> An ECMAscript interpreter would be a big help, too.

Yes, but what is the advantage of having these in the standard library? 
  These seem fairly niche, in that not that many people actually do this 
kind of work; isn't it better to simply organize an effort among the 
interested parties?

That said, I think it would be perhaps an appropriate middleground for 
the standard library documentation to refer to specific projects.

>>- Asynchronous fetch
>>
>>   Not sure how this would work.  What kind of async networking can we 
>>do in a cross-platform manner?
> 
> 
> I ran into this building the Plucker web spider.  You need to be able
> to issue requests for web pages (or other resources) and come back
> from time to time to see if they had loaded yet.  I think the addition
> of generators might be useful here.

Yes, but there's still the cross-platform asynchronous socket handling 
issues.  These seem to require fiddling and details that can't be 
responded to well in the standard library.  Or, at least, this has to be 
built on a lower-level network layer that (AFAIK) does not exist in the 
standard library.

>>- Connection caching
>>
>>   Seems somewhat complex.  I'm not sure it is appropriate for the 
>>standard library.  Maybe httplib2 or another such project could take 
>>this on, but it doesn't seem like a standard library task.
> 
> 
> My philosophy of the standard library is that it should keep working
> when you lean on it, not break and force you to find something else
> when you lean on it.  Apparently (and oddly) that seems to be a
> somewhat unusual way of thinking about it.

The standard library only gets substantial updates every year or two, 
and there's no real user feedback process (except predictive in the case 
of a PEP, or by way of using an already mature library).

>>- Server-side SSL support
>>
>>   What is the current state of this?  It's mostly there, isn't it?
> 
> 
> Not there at all, as far as I know.  The standard library only
> supports client-side SSL.

I suppose the problem is the certificate management, or something along 
those lines?  It's not that hard to do as it is, and I believe Clark has 
specifically mentioned what he would like so that HTTPS would be fairly 
easy to apply to any WSGI server (even if it may not be built in).

>>- A standard server framework on the order of Medusa
>>
>>   Well, we have BaseHTTPServer, and now a WSGI based server.
> 
> 
> So, does either work as well as Medusa?  Surely BaseHTTPServer doesn't.

I dunno.  I'm more concerned that such a server have the same interface 
as a better server could have.  WSGI alone may be enough of a standard 
there.  BaseHTTPServer has been very successful at getting people doing 
things very quickly, in a way that is satisfying and motivating.  It did 
that without being a really robust web server.  I think it's okay that 
people have to move on to other software at a certain point, so long as 
it is not jarring, and the stdlib interface is not so hopelessly naive 
that any "real" alternative feels very different.  I don't think the 
real alternative(s) to BaseHTTPServer have to feel that different.

>>- Explicit cookie handling
>>
>>   I guess I brought this one up?  I'm not sure what I mean ;)
> 
> 
> I had to do some client-side cookie handling lately...  There are some
> missing pieces, like cookie databases that automatically sync with the
> user's browser cookie repositories.

I think cookielib.CookieJar gives you your database, and reads browser 
databases.

> I don't believe there's any support for server-side cookie handling either.

On the server side the Cookie module is crude but workable.  If I was to 
make a request object with a .cookies attribute, it would use but not 
expose any part of that module, since the cookies the server recieves 
can be fully described by a dictionary (unlike the cookies that a server 
sends).

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org