Pimping the 'cgi' module

[robert]

Christoph Haas wrote:
> On Thursday 23 November 2006 21:29, robert wrote:
>> When a LAMP programmer comes to Python, there are so many different
>> confusing things. It starts with a 'non-documented' cgi module - a
>> 'High-Level-Interface', that cannot even iterate over the form items. A
>> name ZOPE in focus which reveals to be a monster system with 2-year
>> learning-curve, ready for running an article editorial system for TOP-10
>> newspaper companies, but unable to make simple things simple. A handful
>> of different Djangos - choose one for each weekday and programmer ...
>> And Ruby made it with this single-known simple URL-to-method-router (And
>> possibly when coming from PHP & Perl one recognizes ::@$_$%§§%/&... and
>> the old namespace dirt) If there would have been a good cgi-system and a
>> somewhat comfortable advanced URL-to-OO-router (beyond the socket
>> wrapper HTTPServer) well exposed in the Python standard lib, the numbers
>> would be probably very different today ...
> 
> I can fully second that. Coming from Perl and being used to the CGI module 
> that is de-facto standard and already doing things much better than 
> Python's equivalent my first thought was: "Does Python really expect me to 
> re-invent the wheel to write basic CGIs?" So I started creating 
> cookie-based session handlers, form field handling etc.

That exactly is where Python lost real numbers. All kinds of fancy things are in the standard lib, but not these must-have's in a good collection ..

> The reason of my posting it not only to complain though. My Python skills 
> are average but I would really like to help contribute some code on this 
> topic. I don't know Ruby-on-Rails myself but I have worked with CGIs for 
> ten years both in C (yuk) and Perl (Perl=semi-yuk / Perl-CGI=yum) and 
> think I know what people coming from Perl expect. And I'm not speaking of 
> Zope or Django or huge frameworks. I try to avoid frameworks wherever I 
> can (I want to write Python - not Zope or Django). I rather rely on the 
> standard library unless it's really saving me time and worth learning 
> another language. Are frameworks perhaps even telling that the standard 
> library is still lacking in some way?

well, note, for that they have named it Ruby-On-Rails, so its still the language - leveraged. 
While it is Zope/Django/Ego-on-Python ... ?

So its about the right level of a framework. (even a cgi module is a framework.)

I think i could learn to like this one as basic intuitive idea: http://www.cherrypy.org/

Unless a Guido'ed version of such thing is not _selected_ into the stdlib or at least promoted single-mindedly and prominently by far before high-tech-but-low-number names like Zope and Django, Python will continue to bleed out heavily on numbers vs. Ruby.

First need of course: an update of that cgi "module".

> I'm thinking of features like:
> - cookie handling
> - cookie-session handling (similar to PHP or Perl's CGI::Session; combined
>   with database backends or simple text files)
> - handling of form fields (Perl's CGI class can easily redisplay what
>   has been entered in the fields when they got submitted through a <FORM>)
> - accessing parameters (honestly I haven't yet understood why I need to use
>   .value on FielStorage dictionaries - why isn't it just a plain
>   dictionary?)
> - state keeping (storing the contents of all form fields on disk
>   to retrieve it later)
> - creating form elements easily (option lists for example from
>   dictionaries and lists like in Perl)
> - creating standard HTML elements (or do you remember how DOCTYPE
>   looks without looking it up?)
> - handling of file uploads (which is just a recipe on ActivePython
>   at the moment)
> - controlling HTTP headers (expires, redirections, charset)
> - convenience functions e.g. for getting the client IP address without
>   needing to read ENV['REMOTE_ADDR']

tell it loud

> Well, of course there are modules like 'cgi' or 'cookielib' and others may 

Cookie. (cookielib is for clients.)
But read the doc chapter "Cookie -- HTTP state management"...
What is 'input': "If input is given, it is passed to the load() method. " ...
So first read the "Example", oohhhh .... :-)

I tell you - you will know how to achieve cookie/session handling before you read all source code and the RFC's

So its not much disadvantage to not know about the existence of this module.

> argue that everything is already there. And there is a 'Web.py' already (I 
> assume every CGI programmer has written their own Web.py already). But 

yes - which Web.py :-)   another v0.138 : http://webpy.org/ ?

> can't we come closer to what other scripting languages provide? Something 
> common? Can't a simple form-based CGI just be a matter of a dozen lines? 
> When it comes to CGIs Python loses its elegance for no reason.

No other language could do it in less lines. There is no other excuse than: *********

> I'm ready to invest work and ideas if anyone else is interested. Perhaps 
> some day it makes it into the standard library or at least gives us 
> something between the features of the standard library and huge framework 
> monsters - something people can be pointed at when starting with Python 
> and CGIs. I'll probably do that anyway because my web projects all use 
> their own reinvented wheel and I'm losing the overview. And even Perl has 
> come that route - it started with a cgi-lib.pl which later became 
> the 'CGI' standard module.

I'd vote for that. 

> Is it just a matter of lacking resources or interest? Even if there is no 

python-dev is fully occupied with top-notch inner life. Of course that is the basis.
But key issues in lib&tools were simply forgotten - left to a random community.

> interest I'll probably read "man CGI" and "man CGI::Session" again and 
> start implementing something nifty for ex-perlies like myself. Hmmm, 
> batteries included even for CGI programmers, a man can dream. :)

Go for a start. In order to realize that essential batteries in good quality within time - even after 10 years now - it is necessary, to hook python-dev for even requesting it actively. Just adding to http://wiki.python.org/moin/WebProgramming and http://wiki.python.org/moin/WebFrameworks is not the task. It requires some organization and somewhat a selection process in addition to good (probably existing) code v0.1xxx material. 
I think it would not be overly complex. Both, a new cgi and possibly included snake "rails" (vs "oil")


Robert