[Web-SIG] Re: Web-SIG Digest, Vol 10, Issue 26

[Peter Hunt]

I believe we can achieve the best of both worlds.

We should implement a Servlet-like interface which works atop WSGI,
which includes session management, caching, and pooling. We should
include this in the standard Python distribution and call it the
official framework.

This servlet library should have the exact same interface as a
currently existing framework. At first glance, I'd say we should just
port jonpy to WSGI and include it in the Python distribution, but
other viable alternatives are WebWare servlets, Snakelets, and
WebStack.

What do you think?

On Thu, 26 Aug 2004 22:23:07 +0200 (CEST), web-sig-request at python.org
<web-sig-request at python.org> wrote:
> Send Web-SIG mailing list submissions to
>        web-sig at python.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>        http://mail.python.org/mailman/listinfo/web-sig
> or, via email, send a message with subject or body 'help' to
>        web-sig-request at python.org
> 
> You can reach the person managing the list at
>        web-sig-owner at python.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Web-SIG digest..."
> 
> Today's Topics:
> 
>   1. Regarding the WSGI draft (Ben Sizer)
>   2. Re: Regarding the WSGI draft (Mark Nottingham)
>   3. Re: Regarding the WSGI draft (Ben Sizer)
>   4. Re: Regarding the WSGI draft (Phillip J. Eby)
>   5. Re: Regarding the WSGI draft (Bob Kimble)
>   6. Re: Regarding the WSGI draft (Titus Brown)
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Thu, 26 Aug 2004 19:30:22 +0100
> From: Ben Sizer <brsizer at kylotan.eidosnet.co.uk>
> Subject: [Web-SIG] Regarding the WSGI draft
> To: web-sig at python.org
> Message-ID: <412E2C3E.7000900 at kylotan.eidosnet.co.uk>
> Content-Type: text/plain; charset=us-ascii; format=flowed
> 
> I've read through the draft and most of the messages on this list that
> followed it. However, I have a basic problem with it which I will
> attempt to summarise below.
> 
> The focus seems to be on making frameworks more portable. The abstract
> reads "This document specifies a proposed standard interface between web
> servers and Python web applications or frameworks, to promote web
> application portability across a variety of web servers." This is all
> well and good, but the implications from that point onwards are that
> we're firmly dealing with frameworks rather than applications. Phillip
> J. Eby has commented on Ian Bicking's blog that "at this stage, the
> benefits of WSGI are primarily for web *framework* authors, and web
> *server* authors, not web *application* authors. This is *not* an
> application API, it's a framework-to-server glue API."
> 
> This immediately strikes me as odd, because from my previous development
> experience frameworks are not that important. In fact, I'm heavily
> inclined to believe that Python only has a proliferation of frameworks
> because of the currently poor degree of higher level support for web
> development in general, and the various frameworks attempt to bridge
> that gap. Create better general web support for Python, and frameworks
> will only be necessary for the really heavy duty applications. Create
> the ability to make frameworks more portable, and all you do is
> encourage more people to develop more frameworks. Focusing on making
> life easier for framework developers is solving the wrong problem, in my
> opinion.
> 
> I come from an ASP and PHP background and generally speaking, a
> developer doesn't want or need a framework between their code and the
> web-scripting language when developing on those platforms. On the rare
> occasions that you do use a framework (such as PHP-Nuke) it's because
> you want to simplify high level activities like news management and user
> lists, and allow people to add content without needing to know HTML. By
> contrast Python's frameworks tend to address the trivial, low level
> things that should fall under the 'batteries included' philosophy that
> Python subscribes to.
> 
> The front page of this Python Web-SIG suggests, "pick a Web framework
> that already exists, make a functionality checklist from it, and add
> that functionality to a new webserver module." I think that's what is
> needed most of all - some sort of standard approach that new Python
> programmers can jump right in and use, which doesn't require choosing
> one of several different frameworks.
> 
> What I'd like to see is something mirroring the Python Database API. For
> instance, I might have to change "import MySQLdb" to "import pyPgSQL"
> but I know that 99% of the rest of the database code will work fine. As
> a web developer I would like to be able to change "import cgi" to
> "import mod_python" or "import fastcgi" and know that, if I follow a
> standard set of calls, I will have a simple and standard way of
> producing a web document. The standardised access to the output and
> input streams in the current draft is all well and good but there's
> little point in me making use of that abstraction if I still have to
> rely on extra modules for access to useful higher-level concepts such as:
> 
> - dispatching control flow based on the URI
> - session management and cookies
> - GET/query string parsing
> - POST/form parsing
> - ASP + PHP style templating
> 
> If these things are coming soon in future WSGI drafts, then great! But I
> got the impression that these features were being delegated out to the
> legion of frameworks.
> 
> I am aware that this all sounds very negative, and I don't mean to
> criticise the hard work that Phillip and others have put into this draft
> specification. I just worry that it diverts attention from what I
> consider to be the real issue facing Python on the web, which is making
> life easier for web application developers, not framework developers.
> 
> --
> Ben Sizer
> 
> ------------------------------
> 
> Message: 2
> Date: Thu, 26 Aug 2004 11:51:06 -0700
> From: Mark Nottingham <mnot at mnot.net>
> Subject: Re: [Web-SIG] Regarding the WSGI draft
> To: Ben Sizer <brsizer at kylotan.eidosnet.co.uk>
> Cc: web-sig at python.org
> Message-ID: <E332D1A0-F790-11D8-82BE-000A95BD86C0 at mnot.net>
> Content-Type: text/plain; charset=US-ASCII; format=flowed
> 
> Hi Ben,
> 
> I understand where you're coming from, but I think we're in a different
> situation here. There are a lot of different ways
> that you can construct an application framework; there is no "one true
> way," because people have varying requirements for a Web application.
> 
> Contrast this with databases, which are for the most part a commodity;
> you can plug in different databases because they all have the same
> conceptual model of how a database works.
> 
> There has been some progress towards convergence on a common view of
> what a Web application is, but I still think we have a ways to go, and
> much to learn, before any one application framework can declare
> victory.
> 
> That being the case, WSGI provides something that's incredibly
> valuable; as long as it maintains the right level of abstraction, it
> allows application frameworks to avoid worrying about the details of a
> particular server implementation.
> 
> I'm pleased as punch with it, because it lets me avoid doing that when
> I write my own application framework (details forthcoming ;).
> 
> Cheers,
> 
> On Aug 26, 2004, at 11:30 AM, Ben Sizer wrote:
> 
> > I've read through the draft and most of the messages on this list that
> > followed it. However, I have a basic problem with it which I will
> > attempt to summarise below.
> >
> > The focus seems to be on making frameworks more portable. The abstract
> > reads "This document specifies a proposed standard interface between
> > web servers and Python web applications or frameworks, to promote web
> > application portability across a variety of web servers." This is all
> > well and good, but the implications from that point onwards are that
> > we're firmly dealing with frameworks rather than applications. Phillip
> > J. Eby has commented on Ian Bicking's blog that "at this stage, the
> > benefits of WSGI are primarily for web *framework* authors, and web
> > *server* authors, not web *application* authors. This is *not* an
> > application API, it's a framework-to-server glue API."
> >
> > This immediately strikes me as odd, because from my previous
> > development experience frameworks are not that important. In fact, I'm
> > heavily inclined to believe that Python only has a proliferation of
> > frameworks because of the currently poor degree of higher level
> > support for web development in general, and the various frameworks
> > attempt to bridge that gap. Create better general web support for
> > Python, and frameworks will only be necessary for the really heavy
> > duty applications. Create the ability to make frameworks more
> > portable, and all you do is encourage more people to develop more
> > frameworks. Focusing on making life easier for framework developers is
> > solving the wrong problem, in my opinion.
> >
> > I come from an ASP and PHP background and generally speaking, a
> > developer doesn't want or need a framework between their code and the
> > web-scripting language when developing on those platforms. On the rare
> > occasions that you do use a framework (such as PHP-Nuke) it's because
> > you want to simplify high level activities like news management and
> > user lists, and allow people to add content without needing to know
> > HTML. By contrast Python's frameworks tend to address the trivial, low
> > level things that should fall under the 'batteries included'
> > philosophy that Python subscribes to.
> >
> > The front page of this Python Web-SIG suggests, "pick a Web framework
> > that already exists, make a functionality checklist from it, and add
> > that functionality to a new webserver module." I think that's what is
> > needed most of all - some sort of standard approach that new Python
> > programmers can jump right in and use, which doesn't require choosing
> > one of several different frameworks.
> >
> > What I'd like to see is something mirroring the Python Database API.
> > For instance, I might have to change "import MySQLdb" to "import
> > pyPgSQL" but I know that 99% of the rest of the database code will
> > work fine. As a web developer I would like to be able to change
> > "import cgi" to "import mod_python" or "import fastcgi" and know that,
> > if I follow a standard set of calls, I will have a simple and standard
> > way of producing a web document. The standardised access to the output
> > and input streams in the current draft is all well and good but
> > there's little point in me making use of that abstraction if I still
> > have to rely on extra modules for access to useful higher-level
> > concepts such as:
> >
> > - dispatching control flow based on the URI
> > - session management and cookies
> > - GET/query string parsing
> > - POST/form parsing
> > - ASP + PHP style templating
> >
> > If these things are coming soon in future WSGI drafts, then great! But
> > I got the impression that these features were being delegated out to
> > the legion of frameworks.
> >
> > I am aware that this all sounds very negative, and I don't mean to
> > criticise the hard work that Phillip and others have put into this
> > draft specification. I just worry that it diverts attention from what
> > I consider to be the real issue facing Python on the web, which is
> > making life easier for web application developers, not framework
> > developers.
> >
> > --
> > Ben Sizer
> > _______________________________________________
> > Web-SIG mailing list
> > Web-SIG at python.org
> > Web SIG: http://www.python.org/sigs/web-sig
> > Unsubscribe:
> > http://mail.python.org/mailman/options/web-sig/mnot%40mnot.net
> >
> 
> --
> Mark Nottingham     http://www.mnot.net/
> 
> ------------------------------
> 
> Message: 3
> Date: Thu, 26 Aug 2004 20:46:07 +0100
> From: Ben Sizer <brsizer at kylotan.eidosnet.co.uk>
> Subject: Re: [Web-SIG] Regarding the WSGI draft
> To: Mark Nottingham <mnot at mnot.net>
> Cc: web-sig at python.org
> Message-ID: <412E3DFF.2000605 at kylotan.eidosnet.co.uk>
> Content-Type: text/plain; charset=us-ascii; format=flowed
> 
> Mark Nottingham wrote:
> 
> > I understand where you're coming from, but I think we're in a different
> > situation here. There are a lot of different ways
> > that you can construct an application framework; there is no "one true
> > way," because people have varying requirements for a Web application.
> 
> ....
> 
> > There has been some progress towards convergence on a common view of
> > what a Web application is, but I still think we have a ways to go, and
> > much to learn, before any one application framework can declare victory.
> 
> Although what you say makes sense on the surface, the fact remains that
> technologies such as ASP and PHP are popular and useful because they
> present a simple and standard interface to the user, whether that user
> is writing a 4 line script, a small application, or a large framework
> upon which to base other applications. With Python you seem stuck with
> two equally unappealing options: slow CGI if you want a simple script,
> where simple is relative since you need to fool around with os.environ,
> printing your own headers, etc - or a complex and idiosyncratic
> framework if you want anything non-trivial, but which is often just as
> complex as PHP straight out of the box, except with a much smaller user
> base and generally less documentation.
> 
> For example, you know that $_GET[varName] is going to be the standard
> way of accessing a querystring variable in PHP. Yet in Python it could
> be part of a request.form dictionary, or
> cgi.parse_qs(os.environ['QUERY_STRING']), or
> modpython.util.parse_qs(req.parsed_uri[7]), etc. Yet we know that query
> strings are part of the RFC2396 standard, so why not have a standard
> module or interface to present to the user?
> 
> I don't see any good reason for this sort of variance, except that
> there's a bias towards accommodating these existing frameworks rather
> than enabling simpler applications of the future, and which I think is a
> symptom of the problem rather than part of the solution.
> 
> --
> Ben Sizer.
> 
> ------------------------------
> 
> Message: 4
> Date: Thu, 26 Aug 2004 15:59:07 -0400
> From: "Phillip J. Eby" <pje at telecommunity.com>
> Subject: Re: [Web-SIG] Regarding the WSGI draft
> To: Ben Sizer <brsizer at kylotan.eidosnet.co.uk>, web-sig at python.org
> Message-ID: <5.1.1.6.0.20040826150415.02cb0d80 at mail.telecommunity.com>
> Content-Type: text/plain; charset="us-ascii"; format=flowed
> 
> At 07:30 PM 8/26/04 +0100, Ben Sizer wrote:
> >I just worry that it diverts attention from what I consider to be the real
> >issue facing Python on the web, which is making life easier for web
> >application developers, not framework developers.
> 
> Unfortunately, every effort to date to create a "framework to end all
> frameworks" has simply resulted in the existence of framework
> N+1.  Why?  Because the creation of a *new* framework means that there is
> no existing code that uses it.  And if the framework only provides features
> that others already have, there's no compelling reason to switch.
> 
> Any approach that ignores the economic reality of present-day Python web
> apps, and provides no way for them to migrate gradually to a new standard,
> is doomed to niche status at best.  (Comparison to ASP and PHP is
> misleading: both had standards for dispatching, sessions, cookies, form
> parsing, and templating *when they were created*, so there was no legacy
> codebase using alternative solutions that had to be migrated.)
> 
> And so, the only way we're going to "steal" the marketshare of existing
> frameworks is with the consent and co-operation of the developers of those
> frameworks.  That means there has to be enough benefit for them to justify
> the effort of getting on board.
> 
> So, please allow me to reveal my top-secret plan for total world
> domination...  :)
> 
> First, the current situation.  Choice of framework is a high investment for
> users, because once they choose, they are stuck with that framework and
> possibly server.  The cost to switch is extremely high.  It's almost as
> though every plumbing manufacturer makes their own sizes of pipes and
> connectors, so once you choose a vendor, you're stuck with them.
> 
> WSGI changes this scenario by introducing competitive pressure to the
> server/framework choice.  As soon as enough framework and server developers
> participate, the others are pushed by network effects to do the
> same.  Users ask, "Why can't I use your framework in any WSGI server?" and
> "Why can't I use any WSGI framework in your server?", pushing the slower
> adopters to either join up or be marginalized.
> 
> But this is just the first phase: standardizing on a size for one kind of
> pipe.  It's not very glamorous, but it fundamentally changes the
> marketplace, and causes many things to appear to spontaneously happen "on
> their own".
> 
> First, users can experiment with other frameworks, especially if those
> frameworks are lightweight.  This builds competitive pressure in the
> direction of lightweight, easy-to-integrate frameworks.  So framework
> developers begin to break their monolithic approaches down into smaller
> pieces that operate on segments of WSGI.  For example, a session service
> that you pass the incoming 'environ' and outgoing 'headers' to, in order
> for it to read and set cookies.  (Notice that this *isn't* a WSGI-defined
> or standardized service, just a service implemented *in terms of* WSGI.)
> 
> Such a service makes little sense to implement today, but people will
> spontaneously begin developing such services once WSGI is a ubiquitous part
> of the Python web development landscape.  It's the most natural thing in
> the world for them to do so, not only because it means a wider audience for
> their service, but because they're likely developing it for a WSGI-based
> environment they're already using.  What other platform would they write it
> for?
> 
> Because these services will be interchangeable to some degree, lock-in is
> limited and competition will determine a winner or winners.  Then, if the
> winners are sufficiently similar to allow useful standardization, that's
> the natural next step.  But, for some services, the differences will be
> important qualitative differences, and standardization would reduce
> meaningful choice.  We don't know in advance what these services should be,
> and we don't know enough to standardize on them now.
> 
> For someone with an ASP or PHP background, that last statement at least
> might sound like sheer lunacy.  But, Python web frameworks have often
> pioneered techniques years ahead of their appearance in ASP, PHP, and Java
> frameworks.  I would hate for us to lose our next great innovation to
> premature standardization.
> 
> But luckily, I don't need to worry: there's simply no way you'll get enough
> Python framework developers (and their users) to agree on such a
> standardization.  For one thing, it's not in their best interests to do
> so.  (Don't let me discourage you from trying, though, if that's what you
> want to do.  I just don't think you'll have much success, and am not
> interested in trying it myself.)
> 
> Anyway, there it is.  My secret plan to fundamentally alter the Python web
> programming universe through secret mind-control market manipulation and
> social engineering.  You found me out.  Now I'll have to kill you.*  :)
> 
> * "And I'd have gotten away with it too, if it hadn't been for those
> meddling kids..."
> 
> (Disclaimer for non-US readers: the above is a humorous reference to an
> American TV cartoon that featured a different character saying this line
> each week, after their nefarious plans were foiled.  It's not me calling
> anybody a meddling kid, or threatening to actually kill anyone!)
> 
> ------------------------------
> 
> Message: 5
> Date: Thu, 26 Aug 2004 16:11:38 -0400
> From: Bob Kimble <rjkimble at alum.mit.edu>
> Subject: Re: [Web-SIG] Regarding the WSGI draft
> To: web-sig at python.org
> Message-ID: <200408261611.38389.rjkimble at alum.mit.edu>
> Content-Type: text/plain;  charset="iso-8859-1"
> 
> On Thursday 26 August 2004 03:46 pm, Ben Sizer wrote:
> > Mark Nottingham wrote:
> > > I understand where you're coming from, but I think we're in a different
> > > situation here. There are a lot of different ways
> > > that you can construct an application framework; there is no "one true
> > > way," because people have varying requirements for a Web application.
> >
> > ...
> >
> > > There has been some progress towards convergence on a common view of
> > > what a Web application is, but I still think we have a ways to go, and
> > > much to learn, before any one application framework can declare victory.
> >
> > Although what you say makes sense on the surface, the fact remains that
> > technologies such as ASP and PHP are popular and useful because they
> > present a simple and standard interface to the user, whether that user
> > is writing a 4 line script, a small application, or a large framework
> > upon which to base other applications. With Python you seem stuck with
> > two equally unappealing options: slow CGI if you want a simple script,
> > where simple is relative since you need to fool around with os.environ,
> > printing your own headers, etc - or a complex and idiosyncratic
> > framework if you want anything non-trivial, but which is often just as
> > complex as PHP straight out of the box, except with a much smaller user
> > base and generally less documentation.
> >
> > For example, you know that $_GET[varName] is going to be the standard
> > way of accessing a querystring variable in PHP. Yet in Python it could
> > be part of a request.form dictionary, or
> > cgi.parse_qs(os.environ['QUERY_STRING']), or
> > modpython.util.parse_qs(req.parsed_uri[7]), etc. Yet we know that query
> > strings are part of the RFC2396 standard, so why not have a standard
> > module or interface to present to the user?
> >
> > I don't see any good reason for this sort of variance, except that
> > there's a bias towards accommodating these existing frameworks rather
> > than enabling simpler applications of the future, and which I think is a
> > symptom of the problem rather than part of the solution.
> 
> I have been reading this thread for a while now, and I haven't commented
> because I have done absolutely no web development using Python. However,
> Mark's comments strike me as being dead on. I'm used to the Java Servlet API,
> which creates an API for servlets and JSP pages. The fact that there are
> several high quality application servers that all support this API suggests
> to me that creating something similar for Python makes a lot of sense. I have
> written JSP's and servlets and run them under Tomcat, but I know that I could
> just as easily run them under WebSphere, WebLogic, JRun, or any others that
> support the API. It seems to me that creating a similar API for Python would
> be terrific. Of course, somebody would also have to write an application
> server to support the API, but I suspect some of the existing frameworks
> could be revamped to support it. Anyway, that's my 2 cents. I would love to
> see something similar to Tomcat and the Java Servlet API for Python.
> 
> ------------------------------
> 
> Message: 6
> Date: Thu, 26 Aug 2004 13:25:10 -0700
> From: Titus Brown <titus at caltech.edu>
> Subject: Re: [Web-SIG] Regarding the WSGI draft
> To: Bob Kimble <rjkimble at alum.mit.edu>
> Cc: web-sig at python.org
> Message-ID: <20040826202510.GA5704 at caltech.edu>
> Content-Type: text/plain; charset=us-ascii
> 
> -> I have been reading this thread for a while now, and I haven't commented
> -> because I have done absolutely no web development using Python. However,
> -> Mark's comments strike me as being dead on. I'm used to the Java Servlet API,
> -> which creates an API for servlets and JSP pages. The fact that there are
> -> several high quality application servers that all support this API suggests
> -> to me that creating something similar for Python makes a lot of sense. I have
> -> written JSP's and servlets and run them under Tomcat, but I know that I could
> -> just as easily run them under WebSphere, WebLogic, JRun, or any others that
> -> support the API. It seems to me that creating a similar API for Python would
> -> be terrific. Of course, somebody would also have to write an application
> -> server to support the API, but I suspect some of the existing frameworks
> -> could be revamped to support it. Anyway, that's my 2 cents. I would love to
> -> see something similar to Tomcat and the Java Servlet API for Python.
> 
> <delurk>
> 
> I've implemented packages at the adapter level (PyWX), the framework
> level (crud that was never released because I found Quixote first), and
> the content level (based variously on CGI, WebWare, and Quixote).
> 
> I'm moderately skeptical of the short term use of the API being
> developed on this list, because in practice it is relatively easy
> to implement a framework that fits on top of all of the existing
> adapters (CGI, mod_python, etc.)  Medium term, I think it will lead
> to a welcome homogenization of server <--> adapter <--> framework
> interaction, and so I think it's a valuable concept.
> 
> The idea of having a single framework (like Java's "servlets") is, I
> think, silly.  Having implemented sites in several of the existing
> frameworks, it is clear that there are several different ways to
> conceptualize the development of Web sites: the Quixote style and
> the WebWare style are two very distinct examples.  Anything that cuts
> down on the variety of available frameworks is going to restrict the
> options, which is bad.
> 
> However, I think it is incumbent upon the developers and users of the
> different frameworks to clearly distinguish between the various options.
> Right now it is very confusing to me, and I've been developing Web sites
> in Python for 5 years ;).
> 
> I'm very confused as to why you need multiple servlet implementations in
> Java.  Wouldn't one do just as well as 10?  It sounds like having 5
> different implementations of the 'os' module in Python...
> 
> --titus
> 
> ------------------------------
> 
> _______________________________________________
> Web-SIG mailing list
> Web-SIG at python.org
> http://mail.python.org/mailman/listinfo/web-sig
> 
> End of Web-SIG Digest, Vol 10, Issue 26
> ***************************************
>