[Web-SIG] Python Web Modules - Version 0.4.1

[Ian Bicking]

James Gardner wrote:
> Agreed, the modules are fairly CGI-orientated.. and none of the examples 
> show anything clever going on.. but I am keen to refactor them and think 
> they could be easily modified.. even thought I might need some advice! I 
> also think the web modules and the WSGI might make a good fit and there 
> would be no harm in writing the necessary glue so that they could be 
> used in both environments. I am also going to look into how hard it 
> would be getting them working with jython.
> 
> I'm just trying to get my head around the best way of doing things..  My 
> understanding is this: the server is constantly running and calls both 
> the application and any encompassing middleware every time a request is 
> made. This means that for each request the middleware and the 
> application are executed for the request. Consequently there is no speed 
> advantage in moving code from the application to the middleware. The 
> only advantage is that it makes certain bits of code more reusable for 
> other applications.

Correct.

> I can see how the web.database structure or cursor can be moved to the 
> server and passed as environ['web.database.cursor']  and 
> environ['web.database.structure']  objects. 

I'm not sure what the benefit would be?  I'd expect those modules to 
stay as libraries for the application to use, just like they are now.

> (btw it is legal to put 
> objects in the environ dictionary isn't it or are the values expected to 
> be strings?) 

Yes, it is legal.

> but surely there would be no advantage to moving things 
> like the web.cgi object away from the application global namespace 
> because it would have to be reloaded on each request anyway so it might 
> as well exist in the application's global space mightn't it? I guess 
> what I'm asking is: for items that have to be refreshed every request is 
> there a lot to be gained by moving them away from the application's 
> global namespcae?

Well, they *have* to be moved away from the global namespace.  There is 
no global request object in WSGI -- the request is represented with the 
environ dictionary, and it has to be passed around.  If it's global, 
then only one request can be processed at a time.  This would make it 
incompatible with threaded environments.

> Could you possibly be more specific about which areas of the modules you 
> think wouldn't work well with threading and why they wouldn't? I don't 
> expect you've studied the modules too closely but I'm not sure I 
> understand where the difficulties might lie?

To be threadsafe, you have to move anything request-related out of 
global variables.  You don't *have* to be threadsafe; you could simply 
not support threaded environments.  That still leaves a number of other 
environments -- CGI, mod_python, and some others -- but I don't think 
it's a good idea to build in that limitation.

The other issue with your modules is that applications shouldn't be 
scripts.  They should be objects of some sort (possibly including 
functions).  The problem with scripts is that they are awkward to work 
with in Python, as you can't import them.  Because if you import them, 
then the script runs, and if you import it a second time, the script 
*won't* run.  And you *must* support an application being run more than 
one time in the same process.

You could get around this, by creating an application object that reruns 
the script everytime it is called, but I think this is unnecessarily 
difficult, and there are other downsides to using scripts in this style.

>>> Key features include:
>>>
>>> * web.auth     - Identity and identification handling. Users may have
>>>  multiple access levels to multiple applications. Sign in and 
>>> password reminder handling is built in.
>>
>>
>>
>>
>> This could be middleware, though obviously it requires a lot of user 
>> configuration.  If it's middleware you could share a single 
>> authentication system with different WSGI applications.  Some 
>> standardization in this case would be good -- starting with things as 
>> simple as environ['auth.username'] holding the string username.  But 
>> for now there's no standard, so you should use a custom prefix.
> 
> 
> 
> Yes, I guess the auth and session modules could be middleware and I am 
> writing an application to handle the sign in and sign out so that 
> wouldn't need to be included in the middleware, just the current auth 
> status of the user and the access levels making the middleware thinner.

Yes, I think that's about right.  More generally, you might just include 
whatever object represents the user, and depend on the application to 
handle its own permission levels.

>>> * web.error    - Enhanced error handling based on the principles of 
>>> the cgitb module. Plain text or HTML output to a file or browser. 
>>> Custom extension mechanism for email notifications and more.
>>
>>
>>
>> This could also be a piece of middleware.  I feel like it's one of the 
>> more complicated kinds of middleware, but useful.  It could also be a 
>> bit of library code that applications can use, but I'd prefer it as 
>> middleware because you could configure it for multiple applications.
> 
> 
> 
> I quite like this as middleware too, but again it could go in the 
> server.. how do you decide? 

I'd be inclined limit the server to the most basic issues, like 
supporting HTTP or interfacing with a web server, and with the basic 
concurrency issues of responding to multiple requests.  I'd rather leave 
other parts out, unless it's really natural to include them.  Like, you 
might include URL resolution in a server based on mod_python, because 
Apache already has URL resolution.

> I also find that all pages have similar 
> regions like title, breadcrumbs, navigation bar, content.. I was 
> planning on having some sort of templating middleware so that 
> applications didn't have to worry so much about the broad page structure 
> allowing easy theming of sites.

A filtering middleware could make sense here.  Otherwise, it might just 
make sense to think of this as configuration -- you indicate what the 
standard template is, and expect the application to select and fill the 
template appropriately.

> At the moment I'm think of refactoring as follows:
> WSGI - Server:      web.database
>                   web.database.object

What would you gain from putting this in the server, instead of a library?

>                   any global config options
>                      - Middleware:  web.auth
>                   web.session
>                   web.error
>                   theming engine

What's your thinking here?  Would the theming engine work for other 
kinds of WSGI applications, e.g., a Webware application?  If not, then I 
don't think there's any need to put this in the server/middleware.

>                      - Application: Sign in, sign out, change password, 
> password reminder, change access levels etc

Yes, definitely application, though there's a configuration aspect -- 
you'd probably configure the authentication middleware to know where 
some of these things were located.

>       - Library:     web.mail
>                   web.image.graph
>                   web.template