From jjinux at gmail.com Fri Apr 1 10:00:15 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri Apr 1 10:00:17 2005 Subject: [Web-SIG] FastCGI (previously "A query for hosting providers") In-Reply-To: <424C4E9E.8010009@colorstudy.com> References: <42473870.6070502@colorstudy.com> <4249235B.2070707@cherrypy.org> <42498FF8.6030407@colorstudy.com> <424BD761.6030202@cherrypy.org> <424C4E9E.8010009@colorstudy.com> Message-ID: > In theory FastCGI is the way to do all of this, but despite my > best efforts I can never get it to work. Well "best efforts" might > indicate more work than I've actually put into it, but enough effort to > leave me thoroughly annoyed ;) Perhaps off topic, I added FastCGI support to Aquarium in the last release. If you'd like, you can steal the code. Here are the docs: Best Regards, -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From pje at telecommunity.com Fri Apr 1 20:02:15 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Apr 1 19:58:17 2005 Subject: [Web-SIG] FastCGI (previously "A query for hosting providers") In-Reply-To: References: <424C4E9E.8010009@colorstudy.com> <42473870.6070502@colorstudy.com> <4249235B.2070707@cherrypy.org> <42498FF8.6030407@colorstudy.com> <424BD761.6030202@cherrypy.org> <424C4E9E.8010009@colorstudy.com> Message-ID: <5.1.1.6.0.20050401120613.02717070@mail.telecommunity.com> At 12:00 AM 4/1/05 -0800, Shannon -jj Behrens wrote: > > In theory FastCGI is the way to do all of this, but despite my > > best efforts I can never get it to work. Well "best efforts" might > > indicate more work than I've actually put into it, but enough effort to > > leave me thoroughly annoyed ;) > >Perhaps off topic, I added FastCGI support to Aquarium in the last >release. If you'd like, you can steal the code. Here are the docs: > > Perhaps even further off topic, I'll second Ian's comment that getting FastCGI set up is a royal pain, especially if you are having Apache start the processes. mod_fastcgi has quite a few poorly documented and half-broken options that interact with each other in bizarre ways. The situation is somewhat better if you start the processes yourself, as then you only have to deal with a handful of poorly-documented timeout/retry parameters. :) PEAK's "FastCGI supervisor" tool manages forking pre-initialized processes (and killing them off) according to a set of simpler, saner options. I wrote it while at Verio so that we could increase performance relative to having Apache start the processes, and getting rid of the configuration headaches was a big plus. Anyway, PEAK's FastCGI supervisor can run any WSGI application, so if you have a high-load application that needs multiple processes to take full advantage of your server hardware, but you don't want to have as many processes as mod_python would create, you should take a look at it. From ianb at colorstudy.com Fri Apr 1 20:19:31 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 1 20:21:12 2005 Subject: [Web-SIG] FastCGI (previously "A query for hosting providers") In-Reply-To: <5.1.1.6.0.20050401120613.02717070@mail.telecommunity.com> References: <424C4E9E.8010009@colorstudy.com> <42473870.6070502@colorstudy.com> <4249235B.2070707@cherrypy.org> <42498FF8.6030407@colorstudy.com> <424BD761.6030202@cherrypy.org> <424C4E9E.8010009@colorstudy.com> <5.1.1.6.0.20050401120613.02717070@mail.telecommunity.com> Message-ID: <424D90B3.6090805@colorstudy.com> Phillip J. Eby wrote: > Perhaps even further off topic, I'll second Ian's comment that getting > FastCGI set up is a royal pain, especially if you are having Apache > start the processes. mod_fastcgi has quite a few poorly documented and > half-broken options that interact with each other in bizarre ways. The > situation is somewhat better if you start the processes yourself, as > then you only have to deal with a handful of poorly-documented > timeout/retry parameters. :) PEAK's "FastCGI supervisor" tool manages > forking pre-initialized processes (and killing them off) according to a > set of simpler, saner options. I wrote it while at Verio so that we > could increase performance relative to having Apache start the > processes, and getting rid of the configuration headaches was a big plus. Yes, it's really the Apache side that I found too difficult, not the Python side. Well, honestly, I had a hard time using your FastCGI server last time I tried it as well -- I don't know which side was broken, but it was altogether more complicated than I was ready for. BTW, there's another FastCGI server here: http://www.saddi.com/software/py-lib/#fcgi In all these cases I think it would be nice to support SCGI in addition to FastCGI -- it nearly the same model (at least if you are ignoring all the complicated FastCGI stuff like starting processes), and it's just so darn much easier to configure. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Fri Apr 1 20:24:24 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 1 20:25:20 2005 Subject: [Web-SIG] A query for hosting providers In-Reply-To: <20050331193606.GC32357@caltech.edu> References: <42473870.6070502@colorstudy.com> <4249235B.2070707@cherrypy.org> <42498FF8.6030407@colorstudy.com> <424BD761.6030202@cherrypy.org> <79990c6b050331103239c0cc6b@mail.gmail.com> <20050331193358.GA32357@caltech.edu> <20050331193606.GC32357@caltech.edu> Message-ID: <424D91D8.7060504@colorstudy.com> Titus Brown wrote: > -> I use relative URLs because of this, but I understand why this might > -> cause you problems. Perhaps we can add a 'root namespace' parameter > -> to WSGI... > > duh. > > That's what SCRIPT_NAME is. BTW, an interesting (I think) pattern that I found, with a hook I added to wsgikit.urlparser which calls __init__.urlparser_hook(environ) when it's parsing the URL. In the root of an "application", I do something like: def urlparser_hook(environ): if 'todo.base_url' not in environ: environ['todo.base_url'] = environ['SCRIPT_NAME'] This way as urlparser continues its parsing I keep a kind of bookmark to the state of SCRIPT_NAME. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From amk at amk.ca Fri Apr 1 20:48:23 2005 From: amk at amk.ca (A.M. Kuchling) Date: Fri Apr 1 20:48:42 2005 Subject: [Web-SIG] FastCGI (previously "A query for hosting providers") In-Reply-To: <424D90B3.6090805@colorstudy.com> References: <424C4E9E.8010009@colorstudy.com> <42473870.6070502@colorstudy.com> <4249235B.2070707@cherrypy.org> <42498FF8.6030407@colorstudy.com> <424BD761.6030202@cherrypy.org> <424C4E9E.8010009@colorstudy.com> <5.1.1.6.0.20050401120613.02717070@mail.telecommunity.com> <424D90B3.6090805@colorstudy.com> Message-ID: <20050401184823.GA27301@rogue.amk.ca> On Fri, Apr 01, 2005 at 12:19:31PM -0600, Ian Bicking wrote: > In all these cases I think it would be nice to support SCGI in addition > to FastCGI -- it nearly the same model (at least if you are ignoring all > the complicated FastCGI stuff like starting processes), and it's just so > darn much easier to configure. In a little thread at http://www.livejournal.com/users/zestyping/101939.html, effbot thinks about creating another framework ("ElementWeb, anyone?"). I thought about that a little -- what would a blank-sheet-of-paper ElementWeb look like? -- but then decided that it's simply not possible. I mean, here are the requirements for how to run applications: 1) Has to be easy to install and configure 2) Has to be installable on hosted services (where you can only upload stuff, not run separate processes) 3) Has to be high-performance. SCGI fails 2); FastCGI fails 2), and may fail 1); standard CGI meets 1) and 2) but fails 3); mod_python can fail 1); running an HTTP server and using proxying fails 2). There's no good solution. (Personaly, I'm happy to discard 2); virtual servers are pretty inexpensive, so I don't think there's any need to limit yourself to FTP-upload-only services.) --amk From ianb at colorstudy.com Fri Apr 1 21:05:27 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 1 21:06:42 2005 Subject: [Web-SIG] FastCGI (previously "A query for hosting providers") In-Reply-To: <20050401184823.GA27301@rogue.amk.ca> References: <424C4E9E.8010009@colorstudy.com> <42473870.6070502@colorstudy.com> <4249235B.2070707@cherrypy.org> <42498FF8.6030407@colorstudy.com> <424BD761.6030202@cherrypy.org> <424C4E9E.8010009@colorstudy.com> <5.1.1.6.0.20050401120613.02717070@mail.telecommunity.com> <424D90B3.6090805@colorstudy.com> <20050401184823.GA27301@rogue.amk.ca> Message-ID: <424D9B77.8010205@colorstudy.com> A.M. Kuchling wrote: > In a little thread at > http://www.livejournal.com/users/zestyping/101939.html, effbot thinks > about creating another framework ("ElementWeb, anyone?"). I thought > about that a little -- what would a blank-sheet-of-paper ElementWeb > look like? -- but then decided that it's simply not possible. > > I mean, here are the requirements for how to run applications: > > 1) Has to be easy to install and configure > 2) Has to be installable on hosted services (where you can only upload stuff, > not run separate processes) > 3) Has to be high-performance. > > SCGI fails 2); FastCGI fails 2), and may fail 1); standard CGI meets > 1) and 2) but fails 3); mod_python can fail 1); running an HTTP server > and using proxying fails 2). There's no good solution. You simply can't have 2 and 3 simultaneously. mod_php is a model, but I don't think mod_python looks like mod_php to hosting providers -- it looks more like mod_perl (which, AFAIK, is not available on commodity hosts). OTOH, it might be reasonable to offer a couple options, including CGI (with a performance penalty tied to application complexity) and some other options like SCGI, FastCGI, HTTP, etc. Each is a compromise of sorts, but if it's easy and reliable to move around that would be okay. In some ways easy HTTP serving makes 2 less important, because cheap hosts are in some ways an area for people to experiment, and an HTTP server makes it easy to experiment on your own computer. It still keeps someone from creating a phpBB-like program for Python and having people throw it up on any little site, but that's not low-hanging fruit at this point. I've been thinking about putting together more server/client glue in WSGIKit. Right now invocation looks like: server.py --server=twisted --webkit-dir=path/to/app I'd like to support more servers and also more frameworks. One server would be "cgi-script", which wouldn't be a server so much as a CGI script builder (where the CGI script would just be the right #! line, some sys.path manipulation, some configuration setup, and the WSGI cgiserver invocation). One can imagine the same thing using one of those all-in-one CGI script builders (mxCGI or something?) that packages everything up into a zip file that can be uploaded to another host. There's nothing magic about the server.py script, of course, except that it doesn't try to be magically pluggable or anything, just hard coding the glue necessary for existing code. I think this less impressive direction is more achievable at this point, and gives a good new-user experience. I'm actually thinking that, given a few simple standards (common configuration, this server invocation script) the proliferation of frameworks isn't so bad as long as we continue to refactor and consolidate the easy parts. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From remi at cherrypy.org Sun Apr 3 12:31:56 2005 From: remi at cherrypy.org (Remi Delon) Date: Sun Apr 3 12:31:55 2005 Subject: [Web-SIG] A query for hosting providers In-Reply-To: <79990c6b050331103239c0cc6b@mail.gmail.com> References: <42473870.6070502@colorstudy.com> <4249235B.2070707@cherrypy.org> <42498FF8.6030407@colorstudy.com> <424BD761.6030202@cherrypy.org> <79990c6b050331103239c0cc6b@mail.gmail.com> Message-ID: <424FC61C.3010006@cherrypy.org> >>The CP-Location trick is not needed (I should remove it from this page >>as it confuses people). > > Hmm, I wrote that part of the page. My specific reason for using a > custom header is that it's the only way I can see to locate a CherryPy > application *not* at the root of a virtual host. Maybe we should continue this discussion on the CherryPy list ... All I was saying is that I didn't want to give people the impression that this custom header was always needed ... It's only needed if you want to locate your app not at the root ... And even then, there are also other alternatives that don't require a custom header. For instance, you can put the path of your application in a config file and use BaseUrlFilter to tell CherryPy where to locate the app. Then you should just relative URLs, or use cpg.request.base for absolute URLs (including redirects). Remi. From remi at cherrypy.org Sun Apr 3 12:46:12 2005 From: remi at cherrypy.org (Remi Delon) Date: Sun Apr 3 12:46:10 2005 Subject: [Web-SIG] A query for hosting providers In-Reply-To: <424C4E9E.8010009@colorstudy.com> References: <42473870.6070502@colorstudy.com> <4249235B.2070707@cherrypy.org> <42498FF8.6030407@colorstudy.com> <424BD761.6030202@cherrypy.org> <424C4E9E.8010009@colorstudy.com> Message-ID: <424FC974.6060700@cherrypy.org> >>> On our own servers we've been using CGI connectors (wkcgi, Zope.cgi), >>> which seem fast enough, and of course won't be crashing Apache. >> >> Yeah, but we wanted a somewhat "standard" way of talking to Apache and >> most frameworks do come with a small HTTP server, so that works fine for >> us and it also completely isolates the process from Apache. > > CGI is pretty standard, isn't it? Are you talking about starting a small CGI script for each request that acts as a gateway between Apache and the other server ? (I think Zope used to call this "presistent CGI"). In that case, then the overhead is not always acceptable. > I think of the adapters as little > pieces of the frameworks themselves. Or just a simpler, more isolated > alternative to mod_*. > >>> Have you looked at Supervisor for long running processes? >>> http://www.plope.com/software/supervisor/ >>> I haven't had a chance to use it, but it looks useful for this sort >>> of thing. >> >> Well, there are several such supervising tools (daemontools is another >> one), but again, they never matched our exact needs. For instance, >> sometimes it's OK if a process is down ... it could just be that the >> user is working on his site. And also, they usually only watch one >> thing: make sure that the process stays up, but there are a million >> other things we wanted to watch for. So we just wrote our own scripts. > > Unlike daemontools, Supervisor is written in Python, which makes it good > ;) It also seems like it's meant ot address just the kind of situation > you're in -- allowing users to restart servers despite having different > permissions, monitoring servers, etc. Thanks for the tip, I'll take a closer look at it then :-) >>> HTTP does seem like a reasonable way to communicate between servers, >>> instead of all these ad hoc HTTP-like protocols (PCGI, SCGI, FastCGI, >>> mod_webkit, etc). My only disappointment with that technique is that >>> you lose some context -- e.g., if REMOTE_USER is set, or >>> SCRIPT_NAME/PATH_INFO (you probably have to configure your URLs, >>> since they aren't detectable), mod_rewrite's additional environmental >>> variables, etc. Hmm... I notice you use custom headers for that >>> (CP-Location), and I suppose other variables could also be passed >>> through... it's just unfortunate because that significantly adds to >>> the Apache configuration, which is something I try to avoid -- it's >>> easy enough to put in place, but hard to maintain. >> >> >> The CP-Location trick is not needed (I should remove it from this page >> as it confuses people). >> Have a look at the section called "What are the drawbacks of running >> CherryPy behind Apache ?" on this page: >> http://www.cherrypy.org/wiki/CherryPyProductionSetup >> It summarizes my view on this (basically, there aren't any real >> drawbacks if you're using mod_rewrite with Apache2). > > > Does Apache 2 add a X-Original-URI header or something? I see the > Forwarded-For and Forwarded-Host headers, but those are only part of the > request -- leaving out REMOTE_USER, SCRIPT_NAME, PATH_INFO, and maybe > some other internal variables. I think you're confusing 2 things... The variables you mention are CGI-specific. I was just talking about mod_rewrite: Apache just forwards the HTTP request it gets to the other server. To this other server, the request will look exactly as if it came from the client, except for 2 things: - The "Host" header will be "localhost" (assuming Apache and the other server are on the same machine) instead of the actual Host header from the original HTTP request (but you can access the original value through X-Forwarded-Host) - The remote IP address will be the one from Apache (again, "localhost" if it's on the same machine). But again, you can access the IP address of the client through X-Forwarded-For - AFAIK, all the other HTTP headers will be kept intact, so it's exactly as if your server was running exposed. I'm just really advocating mod_rewrite between Apache and the other server because that's how I've been running most of my production sites for the past 5 years and it's always worked great :-) Remi. From ianb at colorstudy.com Tue Apr 5 22:40:37 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue Apr 5 22:42:43 2005 Subject: [Web-SIG] A query for hosting providers In-Reply-To: <424FC974.6060700@cherrypy.org> References: <42473870.6070502@colorstudy.com> <4249235B.2070707@cherrypy.org> <42498FF8.6030407@colorstudy.com> <424BD761.6030202@cherrypy.org> <424C4E9E.8010009@colorstudy.com> <424FC974.6060700@cherrypy.org> Message-ID: <4252F7C5.9020904@colorstudy.com> Remi Delon wrote: >>>> On our own servers we've been using CGI connectors (wkcgi, >>>> Zope.cgi), which seem fast enough, and of course won't be crashing >>>> Apache. >>> >>> >>> Yeah, but we wanted a somewhat "standard" way of talking to Apache and >>> most frameworks do come with a small HTTP server, so that works fine for >>> us and it also completely isolates the process from Apache. >> >> >> CGI is pretty standard, isn't it? > > > Are you talking about starting a small CGI script for each request that > acts as a gateway between Apache and the other server ? (I think Zope > used to call this "presistent CGI"). In that case, then the overhead is > not always acceptable. In my Very Light testing Some Years Ago I found the performance of small C CGI scripts to be quite acceptable. This is something like Webware's wkcgi, Zope's Zope.cgi, the fcgi FastCGI script, etc. If I remember correctly, wkcgi was something like 2x slower than mod_webkit for an empty page, and a Python CGI script was at least 10x slower (but I think significantly more). The size of the typical scripts are small enough to be highly cacheable. >>>> HTTP does seem like a reasonable way to communicate between servers, >>>> instead of all these ad hoc HTTP-like protocols (PCGI, SCGI, >>>> FastCGI, mod_webkit, etc). My only disappointment with that >>>> technique is that you lose some context -- e.g., if REMOTE_USER is >>>> set, or SCRIPT_NAME/PATH_INFO (you probably have to configure your >>>> URLs, since they aren't detectable), mod_rewrite's additional >>>> environmental variables, etc. Hmm... I notice you use custom >>>> headers for that (CP-Location), and I suppose other variables could >>>> also be passed through... it's just unfortunate because that >>>> significantly adds to the Apache configuration, which is something I >>>> try to avoid -- it's easy enough to put in place, but hard to maintain. >>> >>> The CP-Location trick is not needed (I should remove it from this page >>> as it confuses people). >>> Have a look at the section called "What are the drawbacks of running >>> CherryPy behind Apache ?" on this page: >>> http://www.cherrypy.org/wiki/CherryPyProductionSetup >>> It summarizes my view on this (basically, there aren't any real >>> drawbacks if you're using mod_rewrite with Apache2). >> >> Does Apache 2 add a X-Original-URI header or something? I see the >> Forwarded-For and Forwarded-Host headers, but those are only part of >> the request -- leaving out REMOTE_USER, SCRIPT_NAME, PATH_INFO, and >> maybe some other internal variables. > > I think you're confusing 2 things... The variables you mention are > CGI-specific. Well, they are also the same variables typically used in most in-Apache systems -- you have them in PHP, in mod_python, via FastCGI, etc. WSGI also makes particular use of SCRIPT_NAME and PATH_INFO, which represents more information than a single URL presents. I find all these variables useful. Many could be represented as extra headers -- including something like CP-Location -- but I find the alternative relies too heavily on configuration for my tastes. > I was just talking about mod_rewrite: Apache just forwards the HTTP > request it gets to the other server. To this other server, the request > will look exactly as if it came from the client, except for 2 things: > - The "Host" header will be "localhost" (assuming Apache and the other > server are on the same machine) instead of the actual Host header from > the original HTTP request (but you can access the original value through > X-Forwarded-Host) > - The remote IP address will be the one from Apache (again, > "localhost" if it's on the same machine). But again, you can access the > IP address of the client through X-Forwarded-For > - AFAIK, all the other HTTP headers will be kept intact, so it's > exactly as if your server was running exposed. > > I'm just really advocating mod_rewrite between Apache and the other > server because that's how I've been running most of my production sites > for the past 5 years and it's always worked great :-) I've used it a lot too, in a variety of ways, and I'm more-or-less comfortable with it. But I think it provides a challenge to the new user, and can be difficult to debug, so I'm looking for clearer alternatives. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Tue Apr 5 22:51:48 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue Apr 5 22:53:00 2005 Subject: [Web-SIG] WSGI start_response exc_info argument In-Reply-To: <5.1.1.6.0.20050330145219.02942e20@mail.telecommunity.com> References: <5.1.1.6.0.20050330141819.034371c0@mail.telecommunity.com> <5.1.1.6.0.20050330114245.03865400@mail.telecommunity.com> <5.1.1.6.0.20050330114245.03865400@mail.telecommunity.com> <5.1.1.6.0.20050330141819.034371c0@mail.telecommunity.com> <5.1.1.6.0.20050330145219.02942e20@mail.telecommunity.com> Message-ID: <4252FA64.8090605@colorstudy.com> Phillip J. Eby wrote: >> But I don't mind all of that, because it is only contained in the >> error catching middleware and no where else. I have other middleware >> that overrides start_response, and don't want to bother with all the >> exc_info in that case. > > > Just pass it through to the upstream start_response; the top-level > server is the only one that needs to care. > > >> And a lot of the logic -- like trying to show errors even when >> there's been a partial response -- is just work, there's no way to get >> around it. > > > So leave it to the server. All I'm saying is that there is no need to > track whether the response has started. It's the server's job to know > that, and the opinion of middleware doesn't count here. As long as the > *server* hasn't sent the headers yet, you can restart the response. My concern is mostly that it is error-prone to leave it to the server, because it's not something you can pass upward easily (AFAICT). I know my middleware is mostly not compliant with this part of the spec, and it's not even clear to me how I'd fix them all. I'm sure I could figure it out, but most of WSGI doesn't require deep thought (and I like that), and this part doesn't feel like that to me. > Therefore, the correct way to send an error is for the error handler to > pass exc_info to start_response, and middleware start_response() > functions *must* pass that upward unless they definitely know better. > (E.g. because they're buffering and know the upstream start_response > hasn't started yet.) > > The point I'm trying to make here is that you seem to be trying to > outsmart WSGI on this point; only the server is in a position to show an > error in the case of a partial response, because it's the only component > that definitively knows what has or hasn't been sent to the client. I'm trying to outsmart the servers, because I want to be able to control the error handling independent of servers. I'm trying to advocate that servers be as dumb as possible, and I expect to trust them as little as possible, so I don't want to leave stuff up to them. And showing partial responses is just Hard -- all the more reason to avoid leaving it up to servers with all the implementations that exist. The error catching middleware is important to me, because I think that's a big part of a compelling web development experience (and something where Python works quite nicely). But I don't expect lots of implementations of error catchers, or at least those implementations *already* require thought. Well, I guess the idea is to let the error middleware do its thing, but give the server an option to bail out gracefully if necessary (by raising the exception passed in). I think it's actually reasonable to have the server bail out ungracefully -- or the middleware -- in those few cases where there's a conflict. It mostly only applies to cases where there's errors in the streamed output, which seems unlikely to me (at least in cases where there's interactive debugging via a web browser). Now that I'm thinking about it, can you remind me why WSGI doesn't work like this: status, headers, body_iter = application(environ) print status print headers... for block in body_iter: ... body_iter.close() Why is there a start_response and then a separate return? -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From pje at telecommunity.com Wed Apr 6 03:26:00 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Apr 6 03:22:45 2005 Subject: [Web-SIG] WSGI start_response exc_info argument In-Reply-To: <4252FA64.8090605@colorstudy.com> References: <5.1.1.6.0.20050330145219.02942e20@mail.telecommunity.com> <5.1.1.6.0.20050330141819.034371c0@mail.telecommunity.com> <5.1.1.6.0.20050330114245.03865400@mail.telecommunity.com> <5.1.1.6.0.20050330114245.03865400@mail.telecommunity.com> <5.1.1.6.0.20050330141819.034371c0@mail.telecommunity.com> <5.1.1.6.0.20050330145219.02942e20@mail.telecommunity.com> Message-ID: <5.1.1.6.0.20050405210154.032365c0@mail.telecommunity.com> At 03:51 PM 4/5/05 -0500, Ian Bicking wrote: >Phillip J. Eby wrote: >>>But I don't mind all of that, because it is only contained in the error >>>catching middleware and no where else. I have other middleware that >>>overrides start_response, and don't want to bother with all the exc_info >>>in that case. >> >>Just pass it through to the upstream start_response; the top-level server >>is the only one that needs to care. >> >>> And a lot of the logic -- like trying to show errors even when >>> there's been a partial response -- is just work, there's no way to get >>> around it. >> >>So leave it to the server. All I'm saying is that there is no need to >>track whether the response has started. It's the server's job to know >>that, and the opinion of middleware doesn't count here. As long as the >>*server* hasn't sent the headers yet, you can restart the response. > >My concern is mostly that it is error-prone to leave it to the server, >because it's not something you can pass upward easily (AFAICT). I don't understand. If you want to implement an in-stream recovery middleware, you certainly can. (But even then you don't need to track the state; if start_response() raises an error, you know the server above you has already sent the headers. So you can always trap the error from start_response in order to *know* that you need in-stream recovery.) And for anything that's *not* in-stream recovery middleware, you shouldn't care; just call start_response with exc_info and proceed about your business. If there is an upstream handler, it will throw exc_info back at you if need be, then catch the error after it breaks out of your code. The purpose of exc_info is to simplify application-level error handlers; they just pass the exc_info and proceed as they would for pre-stream recovery. If pre-stream recovery is impossible, the error handler will be aborted and the server (or error-handling middleware) gets to take over. > I know my middleware is mostly not compliant with this part of the > spec, and it's not even clear to me how I'd fix them all. I'm sure I > could figure it out, but most of WSGI doesn't require deep thought (and I > like that), and this part doesn't feel like that to me. I think you're over-analyzing it and there isn't anything complex except the case of an in-stream handler, which is inherently complex due to the task. But there's nothing stopping you writing middleware that lies to its downstream application when called with exc_info, by *not* re-throwing exc_info but instead attempting recovery. Technically, this is against the letter of the spec, which says that if HTTP headers have been output you must abort. (Although this is then loosened in the Error Handling section to say that error-handling middleware can just return without an exception.) I personally still believe, however, that leaving middleware out of in-streaam recovery is by far the best course of action, because a good framework will buffer its output for the majority of human-readable pages, so in-stream recovery is only needed for streaming data or large files, where *only the application* knows what the safe way to recover is! Therefore, having middleware attempt in-stream recovery is IMO inherently unsafe, unless it is tuned for precisely that particular application, amounting to little more than a monkey patch for that specific scenario. To put it another way, if you think you need this, it's probably because the application isn't buffering properly. In the common case, a WSGI application *should* be sending its output as a single block. (See http://www.python.org/peps/pep-0333.html#buffering-and-streaming for details.) >I'm trying to outsmart the servers, because I want to be able to control >the error handling independent of servers. I'm trying to advocate that >servers be as dumb as possible, and I expect to trust them as little as >possible, so I don't want to leave stuff up to them. And showing partial >responses is just Hard -- all the more reason to avoid leaving it up to >servers with all the implementations that exist. Right; partial responses are hard, so don't do them except in *application* code. 99% of application output should be buffered, so in-stream recovery is irrelevant and useless. >Well, I guess the idea is to let the error middleware do its thing, but >give the server an option to bail out gracefully if necessary (by raising >the exception passed in). I think it's actually reasonable to have the >server bail out ungracefully -- or the middleware -- in those few cases >where there's a conflict. It's allowed to; the spec just says it *should* raise exc_info, but it's allowed to raise something else. > It mostly only applies to cases where there's errors in the streamed > output, which seems unlikely to me (at least in cases where there's > interactive debugging via a web browser). Right, it's only for errors in streamed output that the exc_info argument can even be used; apart from that scenario it's a total red herring. >Now that I'm thinking about it, can you remind me why WSGI doesn't work >like this: > >status, headers, body_iter = application(environ) >print status >print headers... >for block in body_iter: ... >body_iter.close() > > >Why is there a start_response and then a separate return? One reason is that it allows you to write an application as a generator. But more importantly, it's necessary in order to support 'write()' for backward compatibility with existing frameworks, and that's pretty much the "killer reason" it's structured how it is. This particular innovation was Tony Lownds' brainchild, though, not mine. In my original WSGI concept, the application received an output stream and just wrote headers and everything to it. From gvwilson at cs.utoronto.ca Fri Apr 8 14:06:09 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Fri Apr 8 14:13:39 2005 Subject: [Web-SIG] Just lost another one to Rails Message-ID: A colleague down in the States, whom I converted to Python two years ago, sent me mail last night saying, "Well, we've decided to go with Rails. Means learning a new language, but their stuff just plain worked out of the box." He's a bright guy --- used to run an ISP in the mid-90s, and those of you who were involved in Apache between '96 and '01 would probably recognize his name. So my question is, has there been any movement in the wake of Michelle Levesque's web-off, and all the press Rails has been getting, to do something about this? Thanks, Greg From paul at boddie.org.uk Fri Apr 8 14:57:20 2005 From: paul at boddie.org.uk (Paul Boddie) Date: Fri Apr 8 14:38:41 2005 Subject: [Web-SIG] Just lost another one to Rails Message-ID: <200504080557.AA43581502@boddie.org.uk> Greg Wilson wrote: > >So my question is, has there been any movement in the wake of >Michelle Levesque's web-off, and all the press Rails has been >getting, to do something about this? My own plans, time permitting, involve writing clearer introductory documentation for WebStack [1], making a "super bundle" of WebStack together with a few extra packages for typical Web development tasks (probably a forms package I have, plus some XML/XSLT stuff), and giving consideration to making applications easier to deploy than they are at present - that being a general issue with installing vanilla Python applications anyway, as anyone upgrading between Python versions will tell you. In terms of hype, I might post an announcement of any completed activities to comp.lang.python.announce. Echo-chamber effects in the blogsphere, if any, can be produced by other people. Paul [1] http://www.boddie.org.uk/python/WebStack.html ___________________________________________________________ $0 Web Hosting with up to 200MB web space, 1000 MB Transfer 10 Personalized POP and Web E-mail Accounts, and much more. Signup at www.doteasy.com From jjinux at gmail.com Fri Apr 8 15:09:04 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri Apr 8 15:09:06 2005 Subject: [Web-SIG] Just lost another one to Rails In-Reply-To: References: Message-ID: Yep. I'm working my ass off on this with coding, with documenting, and with politics. I know we can do this. It's all a matter of getting people going in the same direction. It's really just a political problem. The code is already almost all there. Best Regards, -jj On Apr 8, 2005 5:06 AM, Greg Wilson wrote: > A colleague down in the States, whom I converted to Python two years > ago, sent me mail last night saying, "Well, we've decided to go with > Rails. Means learning a new language, but their stuff just plain worked > out of the box." He's a bright guy --- used to run an ISP in the > mid-90s, and those of you who were involved in Apache between '96 and > '01 would probably recognize his name. > > So my question is, has there been any movement in the wake of Michelle > Levesque's web-off, and all the press Rails has been getting, to do > something about this? -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From mike_mp at zzzcomputing.com Fri Apr 8 17:39:36 2005 From: mike_mp at zzzcomputing.com (mike bayer) Date: Fri Apr 8 17:40:00 2005 Subject: [Web-SIG] Just lost another one to Rails In-Reply-To: References: Message-ID: <42039.66.192.34.8.1112974776.squirrel@66.192.34.8> it seems like Rails appeals to that same crowd to whom PHP appeals to, and before that things like ASP and Cold Fusion appealed to....i.e. the "big box of code soup that makes small and simple things simple...and big hard things....ah im sure it can do that too". is Python ever going to attract that bunch (and do we want it to) ? > Yep. I'm working my ass off on this with coding, with documenting, > and with politics. I know we can do this. It's all a matter of > getting people going in the same direction. It's really just a > political problem. The code is already almost all there. > > Best Regards, > -jj > > On Apr 8, 2005 5:06 AM, Greg Wilson wrote: >> A colleague down in the States, whom I converted to Python two years >> ago, sent me mail last night saying, "Well, we've decided to go with >> Rails. Means learning a new language, but their stuff just plain worked >> out of the box." He's a bright guy --- used to run an ISP in the >> mid-90s, and those of you who were involved in Apache between '96 and >> '01 would probably recognize his name. >> >> So my question is, has there been any movement in the wake of Michelle >> Levesque's web-off, and all the press Rails has been getting, to do >> something about this? > From ianb at colorstudy.com Fri Apr 8 20:20:19 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 8 20:21:46 2005 Subject: [Web-SIG] Just lost another one to Rails In-Reply-To: References: Message-ID: <4256CB63.7000504@colorstudy.com> Greg Wilson wrote: > So my question is, has there been any movement in the wake of Michelle > Levesque's web-off, and all the press Rails has been getting, to do > something about this? I've been hard at work on WSGIKit, mostly trying to create that out-of-the-box experience. There's been some progress; when I finish the tutorial (http://svn.w4py.org/WSGIKit/trunk/docs/TodoTutorial.txt -- all doctested) I'm planning on making an announcement and a downloadable bundle, website, etc. I've been meaning to talk it up some more here too. I think the core of the setup -- server startup, filesystem setup, configuration -- is applicable to most frameworks. So I'd like to include support for more frameworks. My desire is that for most Python frameworks setup and installation would involve some package installation (an open but important issue -- maybe just distutils installs, or maybe something simpler), and then: app-setup create framework_name app_directory/ cd app_directory wsgi-server And you get a server; SimpleHTTPServer-based by default, but configurable through a configuration in app_directory/server.conf. The app-setup stuff needs to delegate more to the application templates, but there's no magic there (though loading up pluggable modules is always a pain in Python). Or you can do: wsgi-server --server=cgi > /var/www/cgi-bin/myapp.cgi Or: wsgi-server --server=console --url=/ > sample_page.html And all sorts of options are possible. I've actually been changing my mind about what we as a community need to do (my last Rails post was me thinking through those ideas: http://blog.ianbicking.org/what-really-makes-rails-work.html ), and I feel that diversity isn't so bad if we can just make a compelling infrastructure experience, which I think WSGIKit's setup can provide in a relatively generic way. Once we have that, it's just a matter of hawking our individual frameworks to the world (not so much each other) and seeing how things pan out. And I'm certainly willing to change WSGIKit to accomodate more frameworks (and I'm also very open to giving people commit access to the project) -- I actually think WSGIKit will accomodate other frameworks well right now, but I'm just guessing until some more people try to do so. (I also wouldn't mind changing the name of the project, which is a rather lame name right now, but that's another issue -- maybe just making it rebrandable would be fine too) -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From mike_mp at zzzcomputing.com Fri Apr 8 20:50:06 2005 From: mike_mp at zzzcomputing.com (mike bayer) Date: Fri Apr 8 20:50:30 2005 Subject: [Web-SIG] Just lost another one to Rails In-Reply-To: <4256CB63.7000504@colorstudy.com> References: <4256CB63.7000504@colorstudy.com> Message-ID: <34736.66.192.34.8.1112986206.squirrel@66.192.34.8> is there any effort underway to merge WSGI into mod_python, and/or produce an otherwise apache-embedded WSGI ? i would think thats a top priority. > I've actually been changing my mind about what we as a community need to > do (my last Rails post was me thinking through those ideas: > http://blog.ianbicking.org/what-really-makes-rails-work.html ), and I > feel that diversity isn't so bad if we can just make a compelling > infrastructure experience, which I think WSGIKit's setup can provide in > a relatively generic way. Once we have that, it's just a matter of > hawking our individual frameworks to the world (not so much each other) > and seeing how things pan out. > > And I'm certainly willing to change WSGIKit to accomodate more > frameworks (and I'm also very open to giving people commit access to the > project) -- I actually think WSGIKit will accomodate other frameworks > well right now, but I'm just guessing until some more people try to do > so. (I also wouldn't mind changing the name of the project, which is a > rather lame name right now, but that's another issue -- maybe just > making it rebrandable would be fine too) > From ianb at colorstudy.com Fri Apr 8 21:03:41 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 8 21:04:56 2005 Subject: [Web-SIG] Just lost another one to Rails In-Reply-To: <34736.66.192.34.8.1112986206.squirrel@66.192.34.8> References: <4256CB63.7000504@colorstudy.com> <34736.66.192.34.8.1112986206.squirrel@66.192.34.8> Message-ID: <4256D58D.6070807@colorstudy.com> mike bayer wrote: > is there any effort underway to merge WSGI into mod_python, and/or produce > an otherwise apache-embedded WSGI ? i would think thats a top priority. I think Phillip maybe played with such a thing, and maybe someone else did too, but it wasn't clear that someone invested in mod_python has taken it on or supported it. It should be easy, but maybe some mod_python people need to be poked? How it fits into wsgi-server I'm not sure -- probably like CGI it would create a small file that would provide the proper handlers, and a little sample Apache configuration that would have to be put in place manually. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From tony at lownds.com Fri Apr 8 22:33:06 2005 From: tony at lownds.com (tony@lownds.com) Date: Fri Apr 8 22:33:15 2005 Subject: [Web-SIG] setup.py for web apps (not frameworks) Message-ID: <59801.63.227.42.63.1112992386.squirrel@lownds.com> Hi, I'd like to provide a setup.py with web apps I may write. Do any frameworks out there provide distutils extensions for use in a setup.py file? That could be a very useful cross-framework facility... -Tony From floydophone at gmail.com Sat Apr 9 17:20:48 2005 From: floydophone at gmail.com (Peter Hunt) Date: Sat Apr 9 17:20:51 2005 Subject: [Web-SIG] Re: Just lost another one to Rails Message-ID: <6654eac4050409082039afe13b@mail.gmail.com> Ruby on Rails, ColdFusion, ASP.NET , and to a lesser degree PHP and ASP share two important traits that no Python web framework currently embraces. First, when one writes an application for these frameworks, one spends the vast majority of time writing code for their application, writing the logic that their application specifically requires. Contrast this with J2EE or Zope. In writing a Zope 3 application, for example, one must design objects that fit the Zope interface requirements, write a couple of XML configuration files to document the object, and figure out the entire API all at once. Contrast this to PHP, where one spends time simply writing what their application needs to do, and does not need to write a single ambiguous XML configuration file. This extends to deployment. In a J2EE application, you need to deploy a WAR, while with PHP, you just need to drop a few .php files on the server and it works. Second, these frameworks have "batteries included". Rails is a "full-stack" framework, which, according to its API documentation, "includes everything needed to create database-backed web-applications according to the Model-View-Control pattern of separation." It handles everything from form validation to database integration to sending email. No Python framework currently embodies such functionality with such good integration. I really want to be able to say that we should all come together to improve Zope, the "king" of Python web frameworks . . . but I can't say that. Zope 2 was a mess, and Zope 3 is so overengineered that it's painful to write code. The ideal framework should allow the programmer to be organized, while still allowing a monkey to write Hello, World. My two cents . . . what's yours? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20050409/2565a537/attachment.html From mike_mp at zzzcomputing.com Sat Apr 9 20:30:57 2005 From: mike_mp at zzzcomputing.com (michael bayer) Date: Sat Apr 9 20:31:04 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <6654eac4050409082039afe13b@mail.gmail.com> References: <6654eac4050409082039afe13b@mail.gmail.com> Message-ID: <3a7c0e52ac493ed441510d7b0ce8fbf9@zzzcomputing.com> my own project, Myghty, is modeled after Perl's HTML::Mason, which in turn is a lot like PHP with regards to "just plug it in and start writing pages". it does foster a more compentized design than PHP and also integrates nicely into whatever MVC framework the developer chooses. Myghty also includes a rudimentary MVC framework paradigm built into it, but its completely optional; you can use whatever architecture you want and just use the python server pages aspect of the engine. the whole idea with Myghty is that theres a lot of options for how it can be used, it wont let you down when the site gets big, and also its as "out of the box" as it gets; just untar the dist, and it includes a demo server that runs right out of the distribution directory so you can see it run, browse the docs and source code, and start playing with your own pages without even installing it. Basing it off of HTML::Mason was because i think the Mason development model is extremely productive, having used it for several years, after lots of experience with both JSP/servlet and ASP models which I feel are less productive. a lot of other people agree too; its the most popular web framework for Perl, OReilly book and everything. unfortunately I did Myghty a disservice by not showing up to hawk it at Pycon. as its deriviate of something from the perl world, i would never expect it to become the "de-facto" python tool; but then also, i think the python web framework world should remain open to various architectures coexisting. On Apr 9, 2005, at 11:20 AM, Peter Hunt wrote: > Ruby on Rails, ColdFusion, ASP.NET, and to a lesser degree PHP and > ASP share two important traits that no Python web framework currently > embraces. > > > > First, when one writes an application for these frameworks, one > spends the vast majority of time writing code for their application, > writing the logic that their application specifically requires. > Contrast this with J2EE or Zope. In writing a Zope 3 application, for > example, one must design objects that fit the Zope interface > requirements, write a couple of XML configuration files to document > the object, and figure out the entire API all at once. Contrast this > to PHP, where one spends time simply writing what their application > needs to do, and does not need to write a single ambiguous XML > configuration file. This extends to deployment. In a J2EE application, > you need to deploy a WAR, while with PHP, you just need to drop a few > .php files on the server and it works. > > > > Second, these frameworks have "batteries included". Rails is a > "full-stack" framework, which, according to its API documentation, > "includes everything needed to create database-backed web-applications > according to the Model-View-Control pattern of separation." It handles > everything from form validation to database integration to sending > email. No Python framework currently embodies such functionality with > such good integration. > > > > I really want to be able to say that we should all come together to > improve Zope, the "king" of Python web frameworks . . . but I can't > say that. Zope 2 was a mess, and Zope 3 is so overengineered that it's > painful to write code. The ideal framework should allow the programmer > to be organized, while still allowing a monkey to write Hello, World. > > My two cents . . . what's yours? > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/ > mike_mp%40zzzcomputing.com From pje at telecommunity.com Sat Apr 9 21:36:38 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat Apr 9 21:34:14 2005 Subject: [Web-SIG] setup.py for web apps (not frameworks) In-Reply-To: <59801.63.227.42.63.1112992386.squirrel@lownds.com> Message-ID: <5.1.1.6.0.20050409153425.00a2d100@mail.telecommunity.com> At 01:33 PM 04/08/2005 -0700, tony@lownds.com wrote: >Hi, > >I'd like to provide a setup.py with web apps I may write. >Do any frameworks out there provide distutils extensions >for use in a setup.py file? > >That could be a very useful cross-framework facility... FYI, the work I'm currently doing on Python Eggs would probably help with this: http://peak.telecommunity.com/DevCenter/PythonEggs My thought is that if we ever settle on a decent "deployment descriptor" format, you could bundle it inside a .egg and have a single-file deployment option. Eggs will also soon be able to specify library eggs that they depend on, and have the runtime library take care of the rest. From jjinux at gmail.com Mon Apr 11 08:23:01 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Mon Apr 11 08:23:05 2005 Subject: [Web-SIG] (off topic) Re: Just lost another one to Rails In-Reply-To: <3a7c0e52ac493ed441510d7b0ce8fbf9@zzzcomputing.com> References: <6654eac4050409082039afe13b@mail.gmail.com> <3a7c0e52ac493ed441510d7b0ce8fbf9@zzzcomputing.com> Message-ID: That's interesting to hear you mention HTML::Mason. I really liked using it at one company that insisted I use Perl. In fact, I liked the callNext feature so much, I included it as a core part of Aquarium: I even wrote a Dr. Dobb's article about it, which ought to come out before too long. I also liked the templating engine, if you could call it that, although Aquarium uses Cheetah instead. I'm really open to the idea of integrating other templating engines, especially one with Mason syntax. Afterall, templating engines are a matter of taste. Best Regards, -jj On Apr 9, 2005 11:30 AM, michael bayer wrote: > my own project, Myghty, is modeled after Perl's HTML::Mason, which in > turn is a lot like PHP with regards to "just plug it in and start > writing pages". it does foster a more compentized design than PHP and > also integrates nicely into whatever MVC framework the developer > chooses. Myghty also includes a rudimentary MVC framework paradigm > built into it, but its completely optional; you can use whatever > architecture you want and just use the python server pages aspect of > the engine. > > the whole idea with Myghty is that theres a lot of options for how it > can be used, it wont let you down when the site gets big, and also its > as "out of the box" as it gets; just untar the dist, and it includes a > demo server that runs right out of the distribution directory so you > can see it run, browse the docs and source code, and start playing with > your own pages without even installing it. Basing it off of > HTML::Mason was because i think the Mason development model is > extremely productive, having used it for several years, after lots of > experience with both JSP/servlet and ASP models which I feel are less > productive. a lot of other people agree too; its the most popular web > framework for Perl, OReilly book and everything. > > unfortunately I did Myghty a disservice by not showing up to hawk it at > Pycon. as its deriviate of something from the perl world, i would > never expect it to become the "de-facto" python tool; but then also, i > think the python web framework world should remain open to various > architectures coexisting. > > > On Apr 9, 2005, at 11:20 AM, Peter Hunt wrote: > > > Ruby on Rails, ColdFusion, ASP.NET, and to a lesser degree PHP and > > ASP share two important traits that no Python web framework currently > > embraces. > > > > > > > > First, when one writes an application for these frameworks, one > > spends the vast majority of time writing code for their application, > > writing the logic that their application specifically requires. > > Contrast this with J2EE or Zope. In writing a Zope 3 application, for > > example, one must design objects that fit the Zope interface > > requirements, write a couple of XML configuration files to document > > the object, and figure out the entire API all at once. Contrast this > > to PHP, where one spends time simply writing what their application > > needs to do, and does not need to write a single ambiguous XML > > configuration file. This extends to deployment. In a J2EE application, > > you need to deploy a WAR, while with PHP, you just need to drop a few > > .php files on the server and it works. > > > > > > > > Second, these frameworks have "batteries included". Rails is a > > "full-stack" framework, which, according to its API documentation, > > "includes everything needed to create database-backed web-applications > > according to the Model-View-Control pattern of separation." It handles > > everything from form validation to database integration to sending > > email. No Python framework currently embodies such functionality with > > such good integration. > > > > > > > > I really want to be able to say that we should all come together to > > improve Zope, the "king" of Python web frameworks . . . but I can't > > say that. Zope 2 was a mess, and Zope 3 is so overengineered that it's > > painful to write code. The ideal framework should allow the programmer > > to be organized, while still allowing a monkey to write Hello, World. > > > > My two cents . . . what's yours? > > _______________________________________________ > > Web-SIG mailing list > > Web-SIG@python.org > > Web SIG: http://www.python.org/sigs/web-sig > > Unsubscribe: > > http://mail.python.org/mailman/options/web-sig/ > > mike_mp%40zzzcomputing.com > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/jjinux%40gmail.com > -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From faassen at infrae.com Tue Apr 12 15:53:36 2005 From: faassen at infrae.com (Martijn Faassen) Date: Tue Apr 12 15:50:58 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <6654eac4050409082039afe13b@mail.gmail.com> References: <6654eac4050409082039afe13b@mail.gmail.com> Message-ID: <425BD2E0.8000602@infrae.com> Peter Hunt wrote: > In writing a Zope 3 application, for example, one must design objects > that fit the Zope interface requirements, write a couple of XML > configuration files to document the object, and figure out the entire > API all at once. The requirements for a mimimal Zope 3 application are really not that big as you make it sound, though the learning curve could be much improved still. [snip] > I really want to be able to say that we should all come together to improve > Zope, the "king" of Python web frameworks . . . but I can't say that. Zope 2 > was a mess, and Zope 3 is so overengineered that it's painful to write code. I really do not think Zope 3 is "so overengineered that it is painful to write code". Where did you get this idea from? The minimal Zope 3 code is a page template and a few lines of ZCML in a Python package with an empty __init__.py to hook up a new view to an existing object (say, a folder). There's no Python code *at all*, unless you count the expressions in the page template that you might write (or the empty __init__.py). If you want to add some python methods to the view, that's a simple class, and a few more lines of ZCML to hook it up. Writing a simple persistent content object means you need to inherit from Persistent somewhere and add a few more lines of ZCML registration. It's obvious from this that Zope 3 doesn't seem to be accessible to Python programmers as it's supposed to be. I think that's only minimally a technical issue, largely a documentation and presentation issue, and in part I think a historical baggage issue. One issue seems to be that Python programmers are automatically allergic to domain specific glue languages like ZCML, especially when they look like XML. I think this attitude is not really reasonable, but it's extremely widespread. Jim Fulton looked into a Zope 3 based bobo a while back; something I haven't studied, but might be a way to avoid ZCML for some projects. In addition, I think that integrating into the Zope 3 UI, using macros and so on, may be a bit too much of a barrier. Even though it's not that hard, it involves a lot of ZCML here and there before you're fully done. Often, for a web application, you don't care a whit about integrating with the existing Zope UI. It should become clear and obvious how to avoid this. Again, I think mostly a documentation issue, but I think some technical progress might be made there too. > The ideal framework should allow the programmer to be organized, while still > allowing a monkey to write Hello, World. This: http://svn.zope.org/Zope3/trunk/src/zope/app/demo/hellopackage/ is a documented hello world package in Zope 3 by the way. It creates a persistent hello world object that can be added through the Zope 3 UI, with a page template and the like. It's too verbose as a minimal case; dumping the UI requirement and some stylistic issues could cut it down a bit more. If you were to add 'hello world' as a template to an existing object, it'd be a lot simpler still (cutting away all Python code and just leaving a ZCML snippet to hook up the template, and the template itself). It's not monkey-level yet. I think the framework is clean enough to get there. I think it's very important to reach monkey-level. Why don't you hop on over and talk to the Zope 3 developers to try to improve this? Are you sure you're truly dealing with something horribly overengineered here, something that could not be improved at all? I.e, were you actually serious when you said this? > I really want to be able to say that we should all come together to > improve Zope, the "king" of Python web frameworks ... for you seem to have discarded Zope 3 so easily. No questions on a Zope 3 mailing list that I can find, for instance... Regards, Martijn From ianb at colorstudy.com Tue Apr 12 21:15:01 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue Apr 12 21:17:15 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <425BD2E0.8000602@infrae.com> References: <6654eac4050409082039afe13b@mail.gmail.com> <425BD2E0.8000602@infrae.com> Message-ID: <425C1E35.9060902@colorstudy.com> Martijn Faassen wrote: [...] > It's obvious from this that Zope 3 doesn't seem to be accessible to > Python programmers as it's supposed to be. I think that's only minimally > a technical issue, largely a documentation and presentation issue, and > in part I think a historical baggage issue. > > One issue seems to be that Python programmers are automatically allergic > to domain specific glue languages like ZCML, especially when they look > like XML. I think this attitude is not really reasonable, but it's > extremely widespread. Jim Fulton looked into a Zope 3 based bobo a while > back; something I haven't studied, but might be a way to avoid ZCML for > some projects. In my gut -- and I don't think I'm alone -- ZCML doesn't feel right to me. It's quite literally a gut feeling, or maybe you could call it a code smell, it just doesn't smell right to me. Experience might or might not change that feeling, but it's there nonetheless. [...] > This: > > http://svn.zope.org/Zope3/trunk/src/zope/app/demo/hellopackage/ > > is a documented hello world package in Zope 3 by the way. It creates a > persistent hello world object that can be added through the Zope 3 UI, > with a page template and the like. It's too verbose as a minimal case; > dumping the UI requirement and some stylistic issues could cut it down a > bit more. If you were to add 'hello world' as a template to an existing > object, it'd be a lot simpler still (cutting away all Python code and > just leaving a ZCML snippet to hook up the template, and the template > itself). Looking just very briefly, I think the issue is one of approachability and incremental learning. There's some big concepts there, and it's unclear where to start. Interfaces, adaptation, ZCML, ZPT... they are all somewhat foreign, at least to the typical Python programmer. I think Jeffrey Shell has been trying to create a more linear tutorial with a to-do list, which is probably a good idea. A good tutorial -- challenging to make, to be sure -- is one where each step seems clear, relevent to previous steps, and creates something usable. I think this is really where my gut feeling about ZCML comes from -- it's intimately tied to other pieces of the code, and often is a prerequesite to making that code functional (at least functional with respect to its real intention, which is usable in a Zope context), and separate files that have to be subtley in sync always makes me uncomfortable. Of course, that kind of dependency happens all the time, and in an interconnected system you can't expect otherwise. But I won't try to make it an entirely rational argument, that's not actually where my uncomfortable feeling is coming from. I also think some concepts are being overused in Zope 3. Adaptation, specifically -- multi-argument Adapters break my understanding of what Adaptation is supposed to be, and I think they are being used for all sorts of clever, useful things, where "clever" is meant as a pejorative ;) > It's not monkey-level yet. I think the framework is clean enough to get > there. I think it's very important to reach monkey-level. I think monkey-level means you start with the most obvious thing that people understand. I *think* that is HTML, and imperative form- and link-based interaction (though it could be argued that for different audiences this isn't the most obvious place to start). The first thing I see in Zope 3 tends to be objects and interfaces. Maybe this is because Zope 3 seems sexy and interesting when you start from this direction -- you can do exciting things quickly that way -- and people want to show that off. But exciting isn't accessible, and maybe that's a problem. > Why don't you hop on over and talk to the Zope 3 developers to try to > improve this? Are you sure you're truly dealing with something horribly > overengineered here, something that could not be improved at all? I.e, > were you actually serious when you said this? > > > I really want to be able to say that we should all come together to > > improve Zope, the "king" of Python web frameworks > > ... for you seem to have discarded Zope 3 so easily. No questions on a > Zope 3 mailing list that I can find, for instance... Well, I'm not Peter, so you can't call me out about getting involved in Zope 3 since I've made no suggestion I've wanted to ;) But nevertheless, I am interested on some level -- I think there's substantial middle ground where Zope 3 and other Python web projects can come together. On some level Jim's bobo experiments might be this kind of direction, though as just experiments it's hard to say where that might go. A more decoupled Zope 3 might be both more accessible and more flexible. By "decoupled" I don't necessarily mean code changes -- I don't know the details, but I get the impression that what I'm thinking about might be possible right now, that there's been a lot of work to keep Zope 3 decoupled in light of the coupling problems in Zope 2. It's more a matter of perspective, documentation, and packaging. I suspect this would process would be useful inside Zope as well -- for instance, I think there are users that don't want to use the ZODB at all with Zope 3, but right now there's some trivial dependencies on it (like the root object has to be in the ZODB or something). If Zope 3 was factored into separate pieces -- traversal, introspection, ZMI, etc. -- it could be both more usable outside of Zope 3 (allowed for incremental adoption), and more usable inside of Zope 3 for people with specific needs or desires. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From janssen at parc.com Tue Apr 12 21:22:01 2005 From: janssen at parc.com (Bill Janssen) Date: Tue Apr 12 21:22:19 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: Your message of "Tue, 12 Apr 2005 06:53:36 PDT." <425BD2E0.8000602@infrae.com> Message-ID: <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> > The minimal Zope 3 code is a page template and a few lines of ZCML in a > Python package with an empty __init__.py to hook up a new view to an > existing object (say, a folder). There's no Python code *at all* >From my point of view, that's the problem. I don't want to write in some cumbersome and buggy XML format (which is what I'm guessing ZCML is) when I could be writing clean Python code. Bill From carribeiro at gmail.com Tue Apr 12 23:11:42 2005 From: carribeiro at gmail.com (Carlos Ribeiro) Date: Tue Apr 12 23:11:46 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <425C1E35.9060902@colorstudy.com> References: <6654eac4050409082039afe13b@mail.gmail.com> <425BD2E0.8000602@infrae.com> <425C1E35.9060902@colorstudy.com> Message-ID: <864d370905041214114332ac30@mail.gmail.com> On Apr 12, 2005 4:15 PM, Ian Bicking wrote: > I think monkey-level means you start with the most obvious thing that > people understand. I *think* that is HTML, and imperative form- and > link-based interaction (though it could be argued that for different > audiences this isn't the most obvious place to start). The first thing > I see in Zope 3 tends to be objects and interfaces. Maybe this is > because Zope 3 seems sexy and interesting when you start from this > direction -- you can do exciting things quickly that way -- and people > want to show that off. But exciting isn't accessible, and maybe that's > a problem. That sums my feelings about it pretty well. It's not limited to Zope IMHO. There are too many tools in the Web & Python world trying to be too clever. My background is network design, and this kind of discussion reminds me things such as Token Ring x Ethernet, OSI x TCP, and even ATM x TCP/IP to some extent. On one hand we had things that we were 'exciting', well thought out, and made sense in a theorethical sense... in the other corner we had something that was not just as nice, but practical, and it worked out of the box. Who won? In all cases, the most pragmatic design won. It's telling. By the way, I put my two preferred tools on the 'simplicity first' field, and that's CherryPy and SQLObject. It's a shame that I can't help anymore as I am way too busy with my networking projects, but that's my feeling anyway. -- Carlos Ribeiro Consultoria em Projetos blog: http://rascunhosrotos.blogspot.com blog: http://pythonnotes.blogspot.com mail: carribeiro@gmail.com mail: carribeiro@yahoo.com From theman at eradman.com Wed Apr 13 03:11:45 2005 From: theman at eradman.com (Eric Radman) Date: Wed Apr 13 03:16:39 2005 Subject: [Web-SIG] Pure Python HTML? In-Reply-To: <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> References: <425BD2E0.8000602@infrae.com> <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> Message-ID: <20050413011145.GA28928@us270-gl0.eradman.com> On 12:22 Tue 12 Apr , Bill Janssen wrote: > > The minimal Zope 3 code is a page template and a few lines of ZCML in a > > Python package with an empty __init__.py to hook up a new view to an > > existing object (say, a folder). There's no Python code *at all* > > From my point of view, that's the problem. I don't want to write in > some cumbersome and buggy XML format (which is what I'm guessing ZCML > is) when I could be writing clean Python code. I don't know about you, but generating HTML with pure Python code can be messy--ONE reason why we introduce templateing languages in the first place. Often (not always) the best way to end up with XHTML is to start with a valid or almost-valid XML document and then infuse the dynamic content. -- Eric Radman | http://eradman.com From janssen at parc.com Wed Apr 13 04:45:43 2005 From: janssen at parc.com (Bill Janssen) Date: Wed Apr 13 04:46:12 2005 Subject: [Web-SIG] Pure Python HTML? In-Reply-To: Your message of "Tue, 12 Apr 2005 18:11:45 PDT." <20050413011145.GA28928@us270-gl0.eradman.com> Message-ID: <05Apr12.194544pdt."58617"@synergy1.parc.xerox.com> > I don't know about you, but generating HTML with pure Python code can be > messy--ONE reason why we introduce templateing languages in the first > place. Often (not always) the best way to end up with XHTML is to start > with a valid or almost-valid XML document and then infuse the dynamic > content. Indeed. And in Python I do it with string formatting: template = """ %(title)s

%(title)s

Author: %(author)s

something interesting here """ dynamic_content = {} # fill in dynamic content here, or perhaps it's a dict read from a DB dynamic_content['title'] = 'How to write a Web service' dynamic_content['author'] = 'Someone Good' request.reply(template % dynamic_content) Bill From sridharinfinity at gmail.com Wed Apr 13 05:53:54 2005 From: sridharinfinity at gmail.com (Sridhar Ratna) Date: Wed Apr 13 05:53:57 2005 Subject: [Web-SIG] Pure Python HTML? In-Reply-To: <-382823074983716203@unknownmsgid> References: <20050413011145.GA28928@us270-gl0.eradman.com> <-382823074983716203@unknownmsgid> Message-ID: <8816fcf8050412205354171060@mail.gmail.com> > > Indeed. And in Python I do it with string formatting: > > template = """ > > > %(title)s > > >

%(title)s >

Author: %(author)s >

something interesting here > > """ > Or use Nevow stan implementation - http://www.livejournal.com/users/dreid/24711.html -- Sridhar Ratna - http://www.livejournal.com/users/seedar/ From fumanchu at amor.org Wed Apr 13 07:12:14 2005 From: fumanchu at amor.org (Robert Brewer) Date: Wed Apr 13 07:10:19 2005 Subject: [Web-SIG] Pure Python HTML? Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3398795@exchange.hqamor.amorhq.net> Bill Janssen wrote: > > I don't know about you, but generating HTML with pure > Python code can be > > messy--ONE reason why we introduce templateing languages in > the first > > place. Often (not always) the best way to end up with XHTML > is to start > > with a valid or almost-valid XML document and then infuse > the dynamic > > content. > > Indeed. And in Python I do it with string formatting: > > template = """ > > > %(title)s > > >

%(title)s >

Author: %(author)s >

something interesting here > > """ > > dynamic_content = {} > # fill in dynamic content here, or perhaps it's a dict read from a DB > dynamic_content['title'] = 'How to write a Web service' > dynamic_content['author'] = 'Someone Good' > > request.reply(template % dynamic_content) Indeed, indeed, Bill. You might appreciate my Assembly class from Cation: http://www.aminus.org/rbre/cation/html/assembly.py Robert Brewer MIS Amor Ministries fumanchu@amor.org From faassen at infrae.com Wed Apr 13 14:55:55 2005 From: faassen at infrae.com (Martijn Faassen) Date: Wed Apr 13 14:53:13 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <425C1E35.9060902@colorstudy.com> References: <6654eac4050409082039afe13b@mail.gmail.com> <425BD2E0.8000602@infrae.com> <425C1E35.9060902@colorstudy.com> Message-ID: <425D16DB.7070002@infrae.com> Hey, Not much debate from me here on this front, just a lot of agreement. Ian Bicking wrote: > Martijn Faassen wrote: > [...] >> One issue seems to be that Python programmers are automatically >> allergic to domain specific glue languages like ZCML, especially when >> they look like XML. I think this attitude is not really reasonable, >> but it's extremely widespread. Jim Fulton looked into a Zope 3 based >> bobo a while back; something I haven't studied, but might be a way to >> avoid ZCML for some projects. > > In my gut -- and I don't think I'm alone -- ZCML doesn't feel right to > me. It's quite literally a gut feeling, or maybe you could call it a > code smell, it just doesn't smell right to me. Experience might or > might not change that feeling, but it's there nonetheless. Do you think domain specific component configuration/glue languages are not right in general, or is it something specific about ZCML in particular? > [...] > >> This: >> >> http://svn.zope.org/Zope3/trunk/src/zope/app/demo/hellopackage/ >> >> is a documented hello world package in Zope 3 by the way. It creates a >> persistent hello world object that can be added through the Zope 3 UI, >> with a page template and the like. It's too verbose as a minimal case; >> dumping the UI requirement and some stylistic issues could cut it down >> a bit more. If you were to add 'hello world' as a template to an >> existing object, it'd be a lot simpler still (cutting away all Python >> code and just leaving a ZCML snippet to hook up the template, and the >> template itself). > > Looking just very briefly, I think the issue is one of approachability > and incremental learning. Agreed completely, very good point. I harped about this on the Zope 3 list for a while last year. > There's some big concepts there, and it's > unclear where to start. Interfaces, adaptation, ZCML, ZPT... they are > all somewhat foreign, at least to the typical Python programmer. I > think Jeffrey Shell has been trying to create a more linear tutorial > with a to-do list, which is probably a good idea. > A good tutorial -- challenging to make, to be sure -- is one where each > step seems clear, relevent to previous steps, and creates something > usable. And where you don't have to say "ooh, this stuff is something you can ignore for now", *and* where you don't have to explain a lot of irrelevant material as well. My favorite example of this is "Hello world" in Python versus "Hello world" in Java. In Java, to new programmers, you'll either have to say "well, ignore this class thingy for now. Just look at the method. Ignore this too. And that", which gives a bad impression of programming. Or alternatively, you'll have to explain it all "well, this is a class, and that's to do with object oriented programming, which is.." which will confuse and overwhelm new programmers entirely. Zope 3 suffers from the same problem. I think in part this can be solved by writing a good tutorial as you say, and in part we need to ruthlessly eliminate stuff that we otherwise cannot avoid in the tutorial. The Zope 3 tutorial as written by Jim Fulton is very complete, but is not a good tutorial for beginners in my mind. It introduces functional doctesting using tcpwatch before you write your first adapter, or something... I attempted to write something like a simple tutorial in the context of Five (Zope 3 in Zope 2). Here they are, in all their magicpoint glory: http://codespeak.net/svn/z3/Five/trunk/doc/five_interface_tutorial.mgp http://codespeak.net/svn/z3/Five/trunk/doc/five_views_tutorial.mgp Most of this can be safely translated to Zope 3, which in fact I did in part for a presentation once. It is still too overwhelming perhaps for the amount of stuff you end up doing, which isn't that much. > I think this is really where my gut feeling about ZCML comes > from -- it's intimately tied to other pieces of the code, and often is a > prerequesite to making that code functional (at least functional with > respect to its real intention, which is usable in a Zope context), and > separate files that have to be subtley in sync always makes me > uncomfortable. Of course, that kind of dependency happens all the time, > and in an interconnected system you can't expect otherwise. But I won't > try to make it an entirely rational argument, that's not actually where > my uncomfortable feeling is coming from. I recognize the feeling. Python was liberating after having to mess about with header files in C and C++ for me, and they are slightly similar to ZCML. ZCML and interface files bring the "header file" feeling back to Python, which is not good. Even though I also agree that this kind of dependency really happens all the time, and you can't really expect otherwise in larger systems. > I also think some concepts are being overused in Zope 3. Adaptation, > specifically -- multi-argument Adapters break my understanding of what > Adaptation is supposed to be, and I think they are being used for all > sorts of clever, useful things, where "clever" is meant as a pejorative ;) Can't disagree with you there. I don't think a typical user will run into multi-argument adaptation that often though, so I'm not too worried about this. [snip lots more stuff I can only agree with] I agree completely! We need to get this clear to the Zope 3 core hackers though, and I'm not exactly sure how to do that. I mean, I'm sure they understand, but I am also fairly certain it's less high on the priority list than I think it should be. Then again, there is only so much energy available to work on Zope 3 in general, so even if it were high on the list, it might still not make fast progress.. > On some level Jim's bobo experiments might be this kind of direction, > though as just experiments it's hard to say where that might go. A more > decoupled Zope 3 might be both more accessible and more flexible. By > "decoupled" I don't necessarily mean code changes -- I don't know the > details, but I get the impression that what I'm thinking about might be > possible right now, that there's been a lot of work to keep Zope 3 > decoupled in light of the coupling problems in Zope 2. It's more a > matter of perspective, documentation, and packaging. I suspect this > would process would be useful inside Zope as well -- for instance, I > think there are users that don't want to use the ZODB at all with Zope > 3, but right now there's some trivial dependencies on it (like the root > object has to be in the ZODB or something). If Zope 3 was factored into > separate pieces -- traversal, introspection, ZMI, etc. -- it could be > both more usable outside of Zope 3 (allowed for incremental adoption), > and more usable inside of Zope 3 for people with specific needs or desires. Yes, this would be good. It's always been a specific intent of Zope 3, and has been reached in a large part; personally I've been using Zope 3 as a "library" to add some of its facilities to Zope 2 for instance, with quite some success (Five). It needs to be promoted better though, somehow. Zope 3 has a complicated zpackage framework that I don't really like (as I think it's a new wheel that nobody will end up adopting, but I may be wrong), but it's intended to make packaging individual libraries of Zope 3 more easy. I think another attitude shift related to this that might benefit Zope 3 is to become *far* more happy to just reuse other people's code than it's been historically. The whole idea of "well, it needs to be in ZPL and in our repository for us to use it" has got to go somehow. I can see where it is coming from, as it makes maintenance and testing more easy, but being willing to reuse other projects is the flip side of what you're describing. Sometimes I feel like all these Python web frameworks are sitting there and say, there are too many frameworks, why don't you all just use *mine*? Never they consider using code written for some other framework. :) I am exaggarating, but not that much. Zope 3's guilty of that; I did it a bit in my reply to Peter, to try to make a point. :) To get a cluster of reusable components all these frameworks can start using we need to have two sides: * more decoupling of the web frameworks, such as Zope 3 * more willingness by the web frameworks to adopt code not in their control On the good news front, I saw some checkins today indicating Zope 3 is gaining more WSGI capabilities, and even Twisted integration now. That's exactly what we need more of. Anyway, I'll try to take action myself in Zope 3 land to improve the situation where I can, but I only have so much time and too many things to do. :) Regards, Martijn From faassen at infrae.com Wed Apr 13 15:05:18 2005 From: faassen at infrae.com (Martijn Faassen) Date: Wed Apr 13 15:02:34 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> References: <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> Message-ID: <425D190E.2020005@infrae.com> Bill Janssen wrote: >>The minimal Zope 3 code is a page template and a few lines of ZCML in a >>Python package with an empty __init__.py to hook up a new view to an >>existing object (say, a folder). There's no Python code *at all* > > >>From my point of view, that's the problem. I don't want to write in > some cumbersome and buggy XML format (which is what I'm guessing ZCML > is) when I could be writing clean Python code. Buggy? I don't think ZCML is buggy. Where's that coming from? For some tasks Python is *not* the ideal language. I've seen the way we glue up components in Zope 2; either stuff is stored in inaccessible ZODB based databases configured through a lot of bad web user interfaces, or you see a lot of grotty Python code which hacks together component configuration. A domain specific language which does this and only this is a massive improvement. Just the ability to *think* about this problem separately helps. We *weren't* writing clean Python code. I've seen this problem occur over and over in Zope 2 projects. I see this attitude towards domain specific languages all across the python world. People don't like XML document formats either, as they can just use clean, simple, Python datastructures. And lose interoperability, accessibility by a host of programmers that *don't* know your codebase, and code yourself onto an island. In the mean time, in the world of HTML templating, we see a lot more agreement that sometimes a domain specific language is useful. People generally don't want to be producing all their HTML from Python functions. I've seen far less complaints about ZPT being cumbersome and buggy. Possibly there are alternatives to a separate domain specific language in Python that are still clean and maintainable. Can you give me an example of a community of Python developers that has been doing this successfully? That's not say ZCML as it stands doesn't have some didactic/learning curve issues. But to reject this out of hand just like that is a bit too easy, I think. Regards, Martijn From faassen at infrae.com Wed Apr 13 15:12:16 2005 From: faassen at infrae.com (Martijn Faassen) Date: Wed Apr 13 15:09:29 2005 Subject: [Web-SIG] Pure Python HTML? In-Reply-To: <05Apr12.194544pdt."58617"@synergy1.parc.xerox.com> References: <05Apr12.194544pdt."58617"@synergy1.parc.xerox.com> Message-ID: <425D1AB0.4020408@infrae.com> Bill Janssen wrote: >>I don't know about you, but generating HTML with pure Python code can be >>messy--ONE reason why we introduce templateing languages in the first >>place. Often (not always) the best way to end up with XHTML is to start >>with a valid or almost-valid XML document and then infuse the dynamic >>content. > > > Indeed. And in Python I do it with string formatting: [snip] This works for small scale projects where only a few developers are expected to know the codebase. But in a larger scale project where you have to work with web designers which may not know a lot of Python, this doesn't really work. There are also other aspects, like i18ning your HTML, which would be hard to do with your example. It's the black box principle; I don't want to go through your Python code just to tweak a bit of HTML. The idea of ZCML is for programmers to be able to reconfigure or extend the behavior of other people's code without having to change, or hopefully even fully understand, that code itself. The idea is that this pays off once you are working in a larger scale project or cluster of projects, like in the Zope community. I don't think this discussion will go anywhere though, as your position seems to be too extreme in this respect to easily move out of. :) Regards, Martijn From janssen at parc.com Thu Apr 14 02:53:47 2005 From: janssen at parc.com (Bill Janssen) Date: Thu Apr 14 02:54:11 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: Your message of "Wed, 13 Apr 2005 06:05:18 PDT." <425D190E.2020005@infrae.com> Message-ID: <05Apr13.175350pdt."58617"@synergy1.parc.xerox.com> > Buggy? I don't think ZCML is buggy. Where's that coming from? Sorry, didn't mean to knock ZCML specifically. I meant to say that use of XML is inherently buggy when people have to edit it with a text editor, because of the bad syntax. I have the same gripe with the XUL used by Firefox. Nothing specific to the design of ZCML (which I haven't even seen). > And lose > interoperability, accessibility by a host of programmers that *don't* > know your codebase, and code yourself onto an island. Sorry, Martijn, none of these arguments have much weight with me. I'm interested in improving the ability to do things *with Python*. And that doesn't (for me) mean switching to something else, no matter how many other people understand that "something else". Bill From janssen at parc.com Thu Apr 14 03:05:40 2005 From: janssen at parc.com (Bill Janssen) Date: Thu Apr 14 03:06:02 2005 Subject: [Web-SIG] Pure Python HTML? In-Reply-To: Your message of "Wed, 13 Apr 2005 06:12:16 PDT." <425D1AB0.4020408@infrae.com> Message-ID: <05Apr13.180545pdt."58617"@synergy1.parc.xerox.com> > This works for small scale projects where only a few developers are > expected to know the codebase. Sure. It was a small scale example. For larger projects you'd use more abstraction layers, accessing (for example) template strings via method calls which would provide the ability to do things like i18n manipulation. > The idea of ZCML is for programmers to > be able to reconfigure or extend the behavior of other people's code > without having to change, or hopefully even fully understand, that code > itself. Sound engineering principles, modularity and abstraction. Now let's glue those modules together with Python rather than with XML. > I don't think this discussion will go anywhere though, as your position > seems to be too extreme in this respect to easily move out of. :) Gosh, I barely have a position on this, really. I'm just interested, on this mailing list, in improving ways of helping Python-savvy engineers provide and use Web services. Bill From shalabh at cafepy.com Wed Apr 13 07:33:57 2005 From: shalabh at cafepy.com (Shalabh Chaturvedi) Date: Thu Apr 14 03:34:02 2005 Subject: [Web-SIG] Re: Pure Python HTML? In-Reply-To: <20050413011145.GA28928@us270-gl0.eradman.com> References: <425BD2E0.8000602@infrae.com> <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> <20050413011145.GA28928@us270-gl0.eradman.com> Message-ID: Eric Radman wrote: > On 12:22 Tue 12 Apr , Bill Janssen wrote: > >>>The minimal Zope 3 code is a page template and a few lines of ZCML in a >>>Python package with an empty __init__.py to hook up a new view to an >>>existing object (say, a folder). There's no Python code *at all* >> >>From my point of view, that's the problem. I don't want to write in >>some cumbersome and buggy XML format (which is what I'm guessing ZCML >>is) when I could be writing clean Python code. > > > I don't know about you, but generating HTML with pure Python code can be > messy--ONE reason why we introduce templateing languages in the first > place. Often (not always) the best way to end up with XHTML is to start > with a valid or almost-valid XML document and then infuse the dynamic > content. > On the other hand, I find Quixote's PTL[1] very useful. It is very close to Python and automatically escapes problem-chars (like '<'). Also I can factor and reuse parts of a page just like I do with code. Cheers, Shalabh [1] http://www.mems-exchange.org/software/quixote/Quixote-2.0.tar.gz/Quixote-2.0/doc/PTL.txt From paul at boddie.org.uk Thu Apr 14 15:58:00 2005 From: paul at boddie.org.uk (Paul Boddie) Date: Thu Apr 14 15:39:13 2005 Subject: [Web-SIG] ZCML and Zope 3 Message-ID: <200504140658.AA1505558724@boddie.org.uk> Martijn Faassen wrote: > >Buggy? I don't think ZCML is buggy. Where's that coming from? I'd guess that what may have been meant by "buggy" is the observation that by introducing a configuration language alongside the actual work-performing system components, there may be the chance of subtle bugs and misbehaviour arising from difficulties synchronising the ZCML descriptions of the components and the components themselves. Not that I've looked deeply into ZCML, however, but I can sort of understand people when they say that Zope 3 has drifted into J2EE territory. >For some tasks Python is *not* the ideal language. I've seen the >way we glue up components in Zope 2; either stuff is stored in >inaccessible ZODB based databases configured through a lot of bad >web user interfaces, or you see a lot of grotty Python code which >hacks together component configuration. A domain specific language >which does this and only this is a massive improvement. Just the >ability to *think* about this problem separately helps. We >*weren't* writing clean Python code. I've seen this problem occur >over and over in Zope 2 projects. I don't doubt it. The Zope 2 configuration hooks produce quite a neat user experience for administrators, but there's a lot of boilerplate to get wrong (and to confuse beginners). One of the most significant innovations that I saw in the Zope 2 world, albeit from outside, was the whole "components" thing which I presume led to the more pervasive use of interfaces in Zope 3. However, for all this cleaning up, I wonder about two things: the optimism about how patient beginners are in following huge tutorials which don't seem to really show that much (even though instilling the test-driven development discipline may be important); and the way Zope 3 is promoted and how visible/accessible its documentation is. As I noted on someone-or-other's blog, people following tutorials get quite confused and impatient if they're required to "stack up" lots of seemingly arbitrary details. And as someone else noted on their blog, the entry point to Zope 3's documentation is something of a case of "beware of the leopard" (as is following such discussions on different blogs, of course). >I see this attitude towards domain specific languages all across >the python world. People don't like XML document formats either, as >they can just use clean, simple, Python datastructures. And lose >interoperability, accessibility by a host of programmers that >*don't* know your codebase, and code yourself onto an island. There's so much to say on this, but not enough time! I think that the principal objection is that since Python is a programming language, and that Python programs are effectively object manipulating and configuring "scripts", Python may be the best tool for the configuration of components. Of course, there's always the issue of whether you can inspect such "configurators" from other technologies if they're written in Python, and there's always the contention that a full-on programming language can cause more trouble than it gives benefits in such narrowly-defined situations. I think it's harder to argue for using Python when expressing bundles of "pure data", although there are always going to be people in the "S-expressions trump XML" camp who will keep the argument going. Paul ___________________________________________________________ $0 Web Hosting with up to 200MB web space, 1000 MB Transfer 10 Personalized POP and Web E-mail Accounts, and much more. Signup at www.doteasy.com From faassen at infrae.com Fri Apr 15 13:00:41 2005 From: faassen at infrae.com (Martijn Faassen) Date: Fri Apr 15 13:00:44 2005 Subject: [Web-SIG] Re: ZCML and Zope 3 In-Reply-To: <200504140658.AA1505558724@boddie.org.uk> References: <200504140658.AA1505558724@boddie.org.uk> Message-ID: <425F9ED9.4050301@infrae.com> Paul Boddie wrote: > Martijn Faassen wrote: > >> Buggy? I don't think ZCML is buggy. Where's that coming from? > > I'd guess that what may have been meant by "buggy" is the observation > that by introducing a configuration language alongside the actual > work-performing system components, there may be the chance of subtle > bugs and misbehaviour arising from difficulties synchronising the > ZCML descriptions of the components and the components themselves. ZCML is not so much descriptive; that's left to interfaces. I can imagine scenarios like this arising, though I haven't run into clear cut cases that I can recall myself. If you moved your configuration to explicit Python code, I don't see how this would change, however. Only if you could somehow derive your configuration from Python code would this be better, but I'd be far more worried about the magic involved in that than about possible skew between ZCML and Python code. > Not that I've looked deeply into ZCML, however, but I can sort of > understand people when they say that Zope 3 has drifted into J2EE > territory. I don't know J2EE well enough to judge that. ZCML is just a language where you can hook things to other things. You can specify which adapter can adapt a certain interface to another, for instance. You can also specify "template 'something.pt' is used to view objects of interface IFoo through the web, under name 'bar.html'". There are also bits about security, and what appears in which menus (if you use Zope 3's ZMI), and special statements to quickly build edit forms from schema information, etc. >> For some tasks Python is *not* the ideal language. [snip] >> We *weren't* writing clean Python code. I've seen this problem >> occur over and over in Zope 2 projects. > > I don't doubt it. The Zope 2 configuration hooks produce quite a neat > user experience for administrators, but there's a lot of boilerplate > to get wrong (and to confuse beginners). One of the most significant > innovations that I saw in the Zope 2 world, albeit from outside, was > the whole "components" thing which I presume led to the more > pervasive use of interfaces in Zope 3. However, for all this cleaning > up, I wonder about two things: > the optimism about how patient beginners are in following huge > tutorials which don't seem to really show that much (even though > instilling the test-driven development discipline may be important); I worry about the same thing exactly. I do not think the Zope 3 tutorial by Jim Fulton is very encouraging to beginners. If he gives the tutorial in real life, it takes two days, and at the beginning of day 2 you're lucky if you write your first functional doctest. Which doesn't give any gratification at all. The two recent Zope 3 books do better hopefully (I have only browsed through both so far). I've tried myself with my Five tutorial, which has the benefit of focusing on that subset of Zope 3 which is usable from Zope 2. I've had some feedback to that which seems to indicate I'm on the right track there. > and the way Zope 3 is promoted and how visible/accessible its > documentation is. Yup, that just really plain sucks. See my comment here, if you haven't already seen it: http://faassen.n--tree.net/blog/view/weblog/2005/04/06/0 Now the developers will probably make the argument that it's not ready for wide usage yet and that we should still promote Zope 2, but I think that's not correct, as the only way we'll make it ready for wide usage is to actually have increased usage, and for that, we need to promote it. :) > As I noted on someone-or-other's blog, people following tutorials get > quite confused and impatient if they're required to "stack up" lots > of seemingly arbitrary details. And as someone else noted on their > blog, the entry point to Zope 3's documentation is something of a > case of "beware of the leopard" (as is following such discussions on > different blogs, of course). Perhaps it was my blog; you might recognize it. :) >> I see this attitude towards domain specific languages all across >> the python world. People don't like XML document formats either, as >> they can just use clean, simple, Python datastructures. And lose >> interoperability, accessibility by a host of programmers that >> *don't* know your codebase, and code yourself onto an island. > > There's so much to say on this, but not enough time! I think that the > principal objection is that since Python is a programming language, > and that Python programs are effectively object manipulating and > configuring "scripts", Python may be the best tool for the > configuration of components. Of course, there's always the issue of > whether you can inspect such "configurators" from other technologies > if they're written in Python, and there's always the contention that > a full-on programming language can cause more trouble than it gives > benefits in such narrowly-defined situations. Yes, I think both issues, both inspectability and risk (because of the wide scope of Python it's tempting to use it wrongly) are good arguments for domain specific languages once every while, let it be for web templating or component configuration. That said, I'm sure a suitably documented Python API would also do the trick. You'd just load a 'configure.py' instead of a 'configure.zcml'. I am not sure whether the complaints would go away though if you did that, and the risk of having this mixed with functional code is large. > I think it's harder to argue for using Python when expressing bundles > of "pure data", although there are always going to be people in the > "S-expressions trump XML" camp who will keep the argument going. And the people in the "Python datastructures trump XML" camp as well. Sometimes they do trump XML, and sometimes they don't, but the "sometimes they don't" seems to be convieniently ignored all too often. Yes, Pythonic APIs are good, but you can have them when manipulating XML as well, and some pain for interoperability and cross-language understandability can be worth it, anyway. Regards, Martijn From gvwilson at cs.utoronto.ca Fri Apr 15 14:16:34 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Fri Apr 15 14:17:48 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <05Apr13.175350pdt."58617"@synergy1.parc.xerox.com> References: <425D190E.2020005@infrae.com> <05Apr13.175350pdt."58617"@synergy1.parc.xerox.com> Message-ID: It's been a week now since my "Just lost another one to Rails" post, in which I said that a buddy of mine down in the States was switching to Ruby, after using Python for two years, because he and his colleagues needed a lightweight, ready-out-of-the-box web app framework. Responses so far seem to fall into several camps: - "I agree completely, that's why I'm adding yet another framework to the mix!" (I'm waiting for someone to stand up at PyCon and say, "Web App People's Front? We're the People's Front of Web Apps!") - Sneering: "Bah---Rails is impure! Unclean! We must keep our Python pure and elegant!" (Yeah... look how well that worked for Scheme. In my experience, most programmers value usefulness over elegance.) - Whistling in the dark. For example, Ian Bicking said, "...diversity isn't so bad if we can just make a compelling infrastructure experience." I respectfully disagree: right now, the diversity in this area is preventing any of the frameworks from becoming mature enough to be credible among the "I need to get it done now" developers I talk to. (Quick, how many copies of "Programming WebWare/Twisted/CherryPy/whatever" or "The WW/T/CP/whatever Cookbook" are on pre-order? Probably 3500 less than the equivalent RonR books.) It also gives the impression of confusion and bickering, which is lethal when you're trying to persuade someone in the commercial world to adopt something that doesn't come with a 1-800 customer support line. - Frank acknowledgment of RonR's strengths (e.g. Peter Hunt's very welcome post --- Peter, I would have thanked you directly, but I didn't have an email address). RonR is proof that new web app frameworks can displace existing tools like PHP. It's also proof that the existence of a lightweight ready- out-of-the-box don't-have-to-install-eleven-packages-to-make-it-work yes-the-tutorials-are-up-to-date no-you-don't-have-to-write-lots-of- idiosyncratic-XML-templates-or-configuration-files framework is important enough that large numbers of programmers will choose (or switch) their language on that basis alone. So, any bets we'll still be moaning about this after PyCon'06? Greg From amk at amk.ca Fri Apr 15 15:12:55 2005 From: amk at amk.ca (A.M. Kuchling) Date: Fri Apr 15 15:13:56 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <425D190E.2020005@infrae.com> <05Apr13.175350pdt."58617"@synergy1.parc.xerox.com> Message-ID: <20050415131255.GF22766@rogue.amk.ca> On Fri, Apr 15, 2005 at 08:16:34AM -0400, Greg Wilson wrote: > So, any bets we'll still be moaning about this after PyCon'06? For some reason, I'm now picturing you wearing long black robes and carrying a scythe. :) --amk From mike_mp at zzzcomputing.com Fri Apr 15 16:10:06 2005 From: mike_mp at zzzcomputing.com (mike bayer) Date: Fri Apr 15 16:10:50 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <425D190E.2020005@infrae.com> <05Apr13.175350pdt."58617"@synergy1.parc.xerox.com> Message-ID: <48237.66.192.34.8.1113574206.squirrel@66.192.34.8> > - "I agree completely, that's why I'm adding yet another framework to > the mix!" (I'm waiting for someone to stand up at PyCon and say, > "Web App People's Front? We're the People's Front of Web Apps!") I think the term "framework" is becoming a little vague as well. there is the notion of "framework" as, the entire front-to-back approach to serve HTTP requests via Python, then there is the architectural approach that is used on top of an existing web API, and then there are templating languages which have varying degrees of "pluggability" into existing systems. I know that I chimed in with Myghty as "well heres what I did!". But really, its 90% a templating language you can use with whatever "framwork" you want, and 10% an architectural approach which you can use more or less of. In all cases it requires a web API of some kind and doesnt try to replace that. And as far as templating languages for Python, I had a great need for it. The only other powerful options for python-embedded-html seemed to be Cheetah and Spyce, both of which did not fit the bill for me. I'm all for WSGI being as much of a "standard" as we should be embracing. But the Python community is a lot more varied than the Ruby one; people are thinking way out in their own boxes and have their preferred way of doing things (i.e. like people who only want to do python-generated HTML)...instead of reacting to and imitating the Ruby community, we should be presenting the world with our own community, where here are our favorite ways of doing web applications, but there are several varieties of how we do it. In the Python world, you have to use your brain a little bit. From paul at boddie.org.uk Fri Apr 15 17:11:49 2005 From: paul at boddie.org.uk (Paul Boddie) Date: Fri Apr 15 16:52:59 2005 Subject: [Web-SIG] Re: Just lost another one to Rails Message-ID: <200504150811.AA65798380@boddie.org.uk> mike bayer wrote: > >I think the term "framework" is becoming a little vague as well. >there is the notion of "framework" as, the entire front-to-back >approach to serve HTTP requests via Python, then there is the >architectural approach that is used on top of an existing web API, >and then there are templating languages which have varying degrees >of "pluggability" into existing systems. Yes, and I firmly believe in "unbundling" templating languages from frameworks. Perhaps it says more about me than the Python community, or more about the Python community than some other community, but it's a real turn off to look at frameworks where special hooks for some preferred template language are foisted on the developer. People have complained about stuff like how hard it is to use XML with Zope (or perhaps it was that there isn't an easy way in), and I'm sure that there are "how to" texts about working with XML in JSP, but templating should be something that developers can make their own decisions about without sacrificing 90% of the benefits of any given framework. I fondly remember the days when one had to call DTML manually in Bobo applications but still had the remotest chance of using it in other applications, and whilst I see the benefit in supporting instantaneous "hello world"-style page publishing on top of a framework, one shouldn't have to get the powertools out to make other technology choices. >I know that I chimed in with Myghty as "well heres what I did!". >But really, its 90% a templating language you can use with whatever >"framwork" you want, and 10% an architectural approach which you >can use more or less of. In all cases it requires a web API of >some kind and doesnt try to replace that. I'd encourage every templating language developer to make their work as independent of any given framework as possible. As I probably said in some blog or other, plain libraries have a great record when it comes to code re-use, and to be able to cleanly integrate a templating language in an application by just importing some modules and feeding objects or functions with data, on whichever framework, would be highly beneficial for everyone concerned. >And as far as templating languages for Python, I had a great need >for it. The only other powerful options for python-embedded-html >seemed to be Cheetah and Spyce, both of which did not fit the bill >for me. Fair enough. No-one should be criticised for writing something which works for them. What is worrying, however, is the trend towards the proliferation of "highly excluding" top-to-bottom solutions where you buy into the whole thing and cannot easily back out of any particular technology choice without substantial re-plumbing. That there's skepticism in the Python community about Rails, which in part seems to advocate the top-to-bottom buy-in, doesn't surprise me a great deal - there wouldn't be as many different Python frameworks if people loved that sort of thing. >I'm all for WSGI being as much of a "standard" as we should be >embracing. But the Python community is a lot more varied than the >Ruby one; people are thinking way out in their own boxes and have >their preferred way of doing things (i.e. like people who only want >to do python-generated HTML)...instead of reacting to and imitating >the Ruby community, we should be presenting the world with our own >community, where here are our favorite ways of doing web >applications, but there are several varieties of how we do it. >In the Python world, you have to use your brain a little bit. Agreed. The emergence of WSGI seemed to signal some kind of realisation that the layers of the "Web applications stack" should be more independent and interoperable, but I think it would be more productive to pursue such an agenda at a higher level - templating and database access are areas where good solutions exist that work with as much of everything else as possible. Choice in itself isn't a bad thing as long as the decision-making process is supported by good documentation and lets people change their mind about mere implementation details. What I'm advocating is this: * That the community provides narrow/thin but *completely separate* components/solutions which offer very well-defined benefits - eg. Web APIs, templating systems, database access layers. These things shouldn't be mixed up in the fundamental system on the pretense of convenience. * That documentation is produced to describe how one plugs these things into each other and how one might go about integrating other functionality into applications. * That genuine solutions for certain styles of application may be made, but not foisted on people from the lowest levels of any given system. Some applications benefit from having .myapp on the end of every Web resource, and by being able to write "hello world" in a text file and have it pumped out dynamically from the server; not all applications do, however. Paul ___________________________________________________________ $0 Web Hosting with up to 200MB web space, 1000 MB Transfer 10 Personalized POP and Web E-mail Accounts, and much more. Signup at www.doteasy.com From foom at fuhm.net Fri Apr 15 16:59:29 2005 From: foom at fuhm.net (James Y Knight) Date: Fri Apr 15 16:59:35 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <425D190E.2020005@infrae.com> <05Apr13.175350pdt."58617"@synergy1.parc.xerox.com> Message-ID: On Apr 15, 2005, at 8:16 AM, Greg Wilson wrote: > It's been a week now since my "Just lost another one to Rails" post, > in which I said that a buddy of mine down in the States was switching > to Ruby, after using Python for two years, because he and his > colleagues needed a lightweight, ready-out-of-the-box web app > framework. Responses so far seem to fall into several camps: And then there's all the people like me who think this whole the-sky-is-falling stuff is silly. From my point of view the people who are promoting ruby the most are python web developers! At pycon you could hardly go five minutes without someone or other distressing about how completely awesome RoR was and how nothing in python could possibly compete oh no. It's really not that awesome, but it does get quite a few things right. That's nice. But then again, I've never been one for mainstream solutions anyways...having run AOLserver for the last 5 years or so, before seeing the light and switching to Twisted, then getting too involved and suckered into overhauling the twisted web server. So excuse me if I just give a big *yawn* to this whole subject and proceed to silently take over the world. *grin* James From ianb at colorstudy.com Fri Apr 15 19:11:51 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 15 19:13:32 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <425D190E.2020005@infrae.com> <05Apr13.175350pdt."58617"@synergy1.parc.xerox.com> Message-ID: <425FF5D7.4090003@colorstudy.com> Greg Wilson wrote: > It's been a week now since my "Just lost another one to Rails" post, > in which I said that a buddy of mine down in the States was switching > to Ruby, after using Python for two years, because he and his > colleagues needed a lightweight, ready-out-of-the-box web app > framework. Responses so far seem to fall into several camps: > > - "I agree completely, that's why I'm adding yet another framework to > the mix!" (I'm waiting for someone to stand up at PyCon and say, > "Web App People's Front? We're the People's Front of Web Apps!") > > - Sneering: "Bah---Rails is impure! Unclean! We must keep our Python > pure and elegant!" (Yeah... look how well that worked for Scheme. > In my experience, most programmers value usefulness over elegance.) > > - Whistling in the dark. For example, Ian Bicking said, "...diversity > isn't so bad if we can just make a compelling infrastructure > experience." I respectfully disagree: right now, the diversity in > this area is preventing any of the frameworks from becoming mature > enough to be credible among the "I need to get it done now" > developers I talk to. (Quick, how many copies of "Programming > WebWare/Twisted/CherryPy/whatever" or "The WW/T/CP/whatever > Cookbook" are on pre-order? Probably 3500 less than the equivalent > RonR books.) Well, I'm not sure I agree with the rest of these criticisms either, but since you quoted me here I'll counter you on this one... We have to deal with where we are now, we can't simply wish we were somewhere else. Where we are now, there's a bunch of frameworks. So... maybe that can be to Python's advantage, both because we can address problems in a lot of different ways, and we can let natural selection refine the choices we have. If this was ten years ago and I had been looking at the state of Python web programming at the time, this is not what I would have done. But it's not ten years ago, and this is where we are, and instead of being sad about what we aren't, we should look for the good parts of what we are and resolve what problems exist. When *I* have tried to play around with different frameworks, I had a surprisingly hard time of it, and that had a lot to do with installation. I don't think anyone can say that we are doing great at that -- different frameworks accomplish it in different ways, both better and worse, but there's little consistency and it's a maintainance and documentation difficulty for everyone, including framework developers. So, there's some low-hanging fruit. Would it be better if there were less frameworks? Sure, but so what? I can't do anything with that. There's like what, a million web programmers out there? I don't know, but whatever it is there's a whole lot. If we try to split up the pie of *Python* web programmers then we won't get far. But that doesn't matter much, because even if we split the pie up into fewer pieces it's still not a very big pie. > It also gives the impression of confusion and bickering, which is > lethal when you're trying to persuade someone in the commercial > world to adopt something that doesn't come with a 1-800 customer > support line. Bickering? Zope has a weird status in the community, and it sometimes gets both fair and unfair criticism. But otherwise I don't see much that I'd call bickering. When someone posted an article to the Webware wiki on how to convert your Webware app to SkunkWeb, no one complained -- frankly I think it was good for Webware in addition to SkunkWeb, because we're not trying to trap anyone. We're not making any money on this stuff (at least directly), we don't need to cajole people into anything they don't want. > - Frank acknowledgment of RonR's strengths (e.g. Peter Hunt's very > welcome post --- Peter, I would have thanked you directly, but I > didn't have an email address). I'm not afraid to copy what I think is good in RoR. I like the generator scripts, for instance, and I've already added that to WSGIKit. I like that they set up test fixtures early, and I want to add that too. But copying more directly is rather boring and wasteful. > RonR is proof that new web app frameworks can displace existing tools > like PHP. It's also proof that the existence of a lightweight ready- > out-of-the-box don't-have-to-install-eleven-packages-to-make-it-work > yes-the-tutorials-are-up-to-date no-you-don't-have-to-write-lots-of- > idiosyncratic-XML-templates-or-configuration-files framework is > important enough that large numbers of programmers will choose (or > switch) their language on that basis alone. > > So, any bets we'll still be moaning about this after PyCon'06? We might be moaning about something, but not the same things. It's been, like, one month since PyCon? Personally I feel like I've made a lot of progress in my own goals in that time. I wouldn't change my assessment of Python web programming yet -- that's contingent on what other people in the community do -- but I don't feel like we're wallowing hopelessly. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Fri Apr 15 19:14:50 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 15 19:16:32 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <425FF5D7.4090003@colorstudy.com> References: <425D190E.2020005@infrae.com> <05Apr13.175350pdt."58617"@synergy1.parc.xerox.com> <425FF5D7.4090003@colorstudy.com> Message-ID: <425FF68A.5060708@colorstudy.com> Ian Bicking wrote: > Well, I'm not sure I agree with the rest of these criticisms either, but > since you quoted me here I'll counter you on this one... It occurs to me that I probably sound a little pissed off in this email, which really isn't the case. I don't mind the criticism, but I also (personally) feel like I see a good direction forward, and I'm excited about that and I'm feeling a bit of an urge to move past soul-seeking and into implementation. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From gvwilson at cs.utoronto.ca Fri Apr 15 20:07:19 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Fri Apr 15 20:08:17 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <20050415131255.GF22766@rogue.amk.ca> References: <425D190E.2020005@infrae.com> <05Apr13.175350pdt."58617"@synergy1.parc.xerox.com> <20050415131255.GF22766@rogue.amk.ca> Message-ID: > For some reason, I'm now picturing you wearing long black robes and > carrying a scythe. :) Ah... another Pratchett fan :-) From gvwilson at cs.utoronto.ca Fri Apr 15 20:08:40 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Fri Apr 15 20:15:57 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <200504150811.AA65798380@boddie.org.uk> References: <200504150811.AA65798380@boddie.org.uk> Message-ID: > Paul Boddie wrote: > ...I firmly believe in "unbundling" templating languages from frameworks. But doesn't that just make more work for the poor sods who are trying to build things? After all, they have to rebundle them, don't they? Greg From mike_mp at zzzcomputing.com Fri Apr 15 20:45:16 2005 From: mike_mp at zzzcomputing.com (mike bayer) Date: Fri Apr 15 20:46:06 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <200504150811.AA65798380@boddie.org.uk> Message-ID: <27032.66.192.34.8.1113590716.squirrel@66.192.34.8> > > Paul Boddie wrote: >> ...I firmly believe in "unbundling" templating languages from >> frameworks. > > But doesn't that just make more work for the poor sods who are trying to > build things? After all, they have to rebundle them, don't they? theres a tradeoff of initial setup vs. user choice in how they generate their output. pushing one specific template approach favors it over all the others, and a lot of good ideas in all the rejected ones get blown away. the python community, being a brainy bunch, has a particularly wide variety of html generation approaches...I think this diversity should not be discouraged. i would compare to Java Server Pages, which are klunky and hard to use, and how every java-based site is built with crappy looking and hard to debug JSP templates, since "thats what you use", despite the existence of other java-based approaches which are pretty much ignored. ROR is a really new thing. Id be curious to see how popular it is a year from now, and if it even manages to work its way onto a real large scale, high-traffic site (and stays there). since ROR is designed for instant gratification and not really at all for scalable architecture (a field where flexible options are crucial), i have a feeling people will be seeking new replacements at that point. From ianb at colorstudy.com Fri Apr 15 20:49:26 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 15 20:51:05 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <200504150811.AA65798380@boddie.org.uk> Message-ID: <42600CB6.1040100@colorstudy.com> Greg Wilson wrote: > > Paul Boddie wrote: > >> ...I firmly believe in "unbundling" templating languages from frameworks. > > > But doesn't that just make more work for the poor sods who are trying to > build things? After all, they have to rebundle them, don't they? I don't think it's too bad. I'm happy with how ZPTKit works, even though ZPT isn't written (at all) with Webware in mind, or vice versa. OTOH, I don't think ZPT with Webware provides a very good experience without the non-trivial amount of glue code in ZPTKit, and that glue is kind of a NxM problem (# of template languages time # of frameworks). But it's not at the top of my list of problems. Other templating languages, like Webware's PSP (which is JSP-like) are harder to generalize like this, because they are more closely tied to the underlying code (servlets in PSP's case). But, maybe ironically, I find an entirely separate templating language easier to work with. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From rtomayko at gmail.com Fri Apr 15 21:23:53 2005 From: rtomayko at gmail.com (Ryan Tomayko) Date: Fri Apr 15 21:24:01 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <42600CB6.1040100@colorstudy.com> References: <200504150811.AA65798380@boddie.org.uk> <42600CB6.1040100@colorstudy.com> Message-ID: <0968fdb63ef46c86e0fd023bfa457976@gmail.com> On Apr 15, 2005, at 2:49 PM, Ian Bicking wrote: > Greg Wilson wrote: >> > Paul Boddie wrote: >>> ...I firmly believe in "unbundling" templating languages from >>> frameworks. >> But doesn't that just make more work for the poor sods who are trying >> to build things? After all, they have to rebundle them, don't they? > > I don't think it's too bad. I'm happy with how ZPTKit works, even > though ZPT isn't written (at all) with Webware in mind, or vice versa. > OTOH, I don't think ZPT with Webware provides a very good experience > without the non-trivial amount of glue code in ZPTKit, and that glue > is kind of a NxM problem (# of template languages time # of > frameworks). But it's not at the top of my list of problems. > > Other templating languages, like Webware's PSP (which is JSP-like) are > harder to generalize like this, because they are more closely tied to > the underlying code (servlets in PSP's case). But, maybe ironically, > I find an entirely separate templating language easier to work with. We're experiencing this problem with Kid, which is framework neutral. I'm finding that the amount of glue code needed for most frameworks is more than I expected. The other issue I'm having with is that it's really hard to provide anything but really simple template examples and other documentation without assuming a certain framework. I can show the basic layout control and substitution features but it's hard to show real world examples of, say, how a basic site should be laid out. Framework specific idioms bleed into the template. I guess this isn't always bad but it makes it hard to talk about templating without declaring a certain framework beforehand. I'd personally love to see a common set of request/response/session objects (a la Paul's webstack) be adopted. I'm beginning to think that separating templates from frameworks can only work well if the glue in between is somewhat standardized. I've never quite understood why everyone wants to maintain these objects themselves anyway. I've looked at Quixote's, CherryPy's, and Webware's implementations of the basic request/response/session objects and I just don't see a whole lot of variance. I'm not looking for templates to be completely portable or anything, I just wish we could talk about and document them without a chapter about what environment it's running under. Ryan From paul at boddie.org.uk Fri Apr 15 21:29:46 2005 From: paul at boddie.org.uk (Paul Boddie) Date: Fri Apr 15 21:40:00 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <27032.66.192.34.8.1113590716.squirrel@66.192.34.8> References: <200504150811.AA65798380@boddie.org.uk> <27032.66.192.34.8.1113590716.squirrel@66.192.34.8> Message-ID: <200504152129.46714.paul@boddie.org.uk> On Friday 15 April 2005 20:45, mike bayer wrote: > > > Paul Boddie wrote: > >> > >> ...I firmly believe in "unbundling" templating languages from > >> frameworks. > > > > But doesn't that just make more work for the poor sods who are trying to > > build things? After all, they have to rebundle them, don't they? First, let me respond to this. There's nothing to stop anyone from packaging a lot of this stuff together, and if the community moves forward with more convenient installation mechanisms, then perhaps no-one will need to package anything together except for extreme convenience. I was arguing against the tendency for framework developers to more or less *glue* template languages onto Web programming APIs - see the quoted text below for a good example. > theres a tradeoff of initial setup vs. user choice in how they generate > their output. pushing one specific template approach favors it over all > the others, and a lot of good ideas in all the rejected ones get blown > away. the python community, being a brainy bunch, has a particularly wide > variety of html generation approaches...I think this diversity should not > be discouraged. Way back, after my frameworks comparison ended up on the PythonInfo Wiki, I tried to start a comparison of "presentation technologies" because I saw this as an important aspect of Web programming. Sadly, such overviews were tidied away after a "refactoring" of that Wiki, and I lost any interest in keeping it up-to-date. However, such a resource would be very important, I think. > i would compare to Java Server Pages, which are klunky and hard to use, > and how every java-based site is built with crappy looking and hard to > debug JSP templates, since "thats what you use", despite the existence of > other java-based approaches which are pretty much ignored. Indeed, Struts - for a long time the Java framework with most mindshare - is only really a rationalisation of JSP best practices, and coincidentally it has enjoyed similar levels of hype (some justified, perhaps) as Rails now does. Perhaps it's also a coincidence that some of the more vocal advocates of Rails are J2EE old hands who have "found" dynamic languages. Paul From wilk-ml at flibuste.net Fri Apr 15 19:03:32 2005 From: wilk-ml at flibuste.net (Wilk) Date: Fri Apr 15 21:55:03 2005 Subject: [Web-SIG] Re: Just lost another one to Rails References: <425D190E.2020005@infrae.com> <05Apr13.175350pdt."58617"@synergy1.parc.xerox.com> <48237.66.192.34.8.1113574206.squirrel@66.192.34.8> Message-ID: <87fyxskm57.fsf@blakie.riol> mike bayer writes: > I'm all for WSGI being as much of a "standard" as we should be embracing. > But the Python community is a lot more varied than the Ruby one; people > are thinking way out in their own boxes and have their preferred way of > doing things (i.e. like people who only want to do python-generated > HTML)...instead of reacting to and imitating the Ruby community, we should > be presenting the world with our own community, where here are our > favorite ways of doing web applications, but there are several varieties > of how we do it. In the Python world, you have to use your brain a little > bit. I'm totaly agree with this. I use python to make office intranet, web games, project management, monitoring, forum... So much think that don't need the same framework, the same server, the same template engine. I had to do my own framework and it was very easy in python :-) Before for the sames things i had to use php, java, C... The slogan of python-for-the-web could be "so easy to do his own framework" ! It was for this that i switched from php for python (did you try to make a serious framework in php !?) It was just difficult for me to decide if i must hack simplehttpserver or choose one server in the jungle. I also had to make my own session layer... I think wsgi adress this and it's very fine. We must just help developpers to make theirs components wsgi compliants, more tutorials, more examples like wsgikit. I personaly don't think we need to follow ruby or php example. When somes frameworks will said "choose a wsgi server from this list" we'll win :-)! -- William - http://flibuste.net From rtomayko at gmail.com Fri Apr 15 22:02:51 2005 From: rtomayko at gmail.com (Ryan Tomayko) Date: Fri Apr 15 22:02:56 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <20050415193457.DE5F7E1@short.latte.ca> References: <20050415193457.DE5F7E1@short.latte.ca> Message-ID: <8c3190f2657b445e5785a5bbc282459e@gmail.com> On Apr 15, 2005, at 3:52 PM, Blake Winton wrote: >> I'd personally love to see a common set of request/response/session >> objects (a la Paul's webstack) be adopted. > > Isn't that what WSGI offers? Perhaps I need to look closer but my assumption after a pass through the PIP a while ago was that it was more of standard way of getting the information to and from the framework ("middleware") and likely not meant to reach the presentation and controllerish code. I'd love to find out that I'm wrong, although I guess I would find the WSGI idioms around accessing request headers/form-encoded data and collecting response information a bit cumbersome compared to the APIs provided by Webstack, Quixote, Webware, CherryPy, etc. Ryan From ianb at colorstudy.com Fri Apr 15 22:10:15 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 15 22:12:28 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <8c3190f2657b445e5785a5bbc282459e@gmail.com> References: <20050415193457.DE5F7E1@short.latte.ca> <8c3190f2657b445e5785a5bbc282459e@gmail.com> Message-ID: <42601FA7.7040106@colorstudy.com> Ryan Tomayko wrote: > On Apr 15, 2005, at 3:52 PM, Blake Winton wrote: >>> I'd personally love to see a common set of request/response/session >>> objects (a la Paul's webstack) be adopted. >> >> Isn't that what WSGI offers? > > Perhaps I need to look closer but my assumption after a pass through the > PIP a while ago was that it was more of standard way of getting the > information to and from the framework ("middleware") and likely not > meant to reach the presentation and controllerish code. Well, at a certain level you could create the request object that you desire given the WSGI environment, and possibly extract that environment from request objects that already exist. However, that leaves an API disconnect that would probably be a bit uncomfortable. Admittedly, for ZPTKit I made a dictionary-like request object to make the transition from Zope easier (and because dictionary-like requests are easier to use). I would like to see a more limited set of request object APIs, but you can't really do that in a backward-compatible way. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Fri Apr 15 22:17:15 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 15 22:19:18 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <0968fdb63ef46c86e0fd023bfa457976@gmail.com> References: <200504150811.AA65798380@boddie.org.uk> <42600CB6.1040100@colorstudy.com> <0968fdb63ef46c86e0fd023bfa457976@gmail.com> Message-ID: <4260214B.50203@colorstudy.com> Ryan Tomayko wrote: > We're experiencing this problem with Kid, which is framework neutral. > I'm finding that the amount of glue code needed for most frameworks is > more than I expected. > > The other issue I'm having with is that it's really hard to provide > anything but really simple template examples and other documentation > without assuming a certain framework. I can show the basic layout > control and substitution features but it's hard to show real world > examples of, say, how a basic site should be laid out. In a way, I think this is up to the framework -- frameworks tend to have opinions or policy on how files and code is layed out. Templates have opinions on how site look is separate from the layout of individual pages, and stuff like that. Ideally, for a significant class of frameworks it would be nice if they could all be interfaced in the same way. I.e., you pass in a namespace, and maybe a "template space" as well, when templates look up up other templates (as with ZPT macros, standard headers/footers in other other languages, etc). There's small but subtle differences between templates, but maybe those could be papered over. Then docs would be two-fold -- the framework would describe how it loads and populates templates (generically) and templates would describe what they do with the values given to them. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From bwinton at latte.ca Fri Apr 15 22:31:25 2005 From: bwinton at latte.ca (Blake Winton) Date: Fri Apr 15 22:31:30 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <8c3190f2657b445e5785a5bbc282459e@gmail.com> Message-ID: <20050415201419.404A2C3@short.latte.ca> > >> I'd personally love to see a common set of request/response/session > >> objects (a la Paul's webstack) be adopted. > > Isn't that what WSGI offers? > Perhaps I need to look closer but my assumption after a pass through > the PIP a while ago was that it was more of standard way of > getting the information to and from the framework ("middleware") and > likely not meant to reach the presentation and controllerish code. Hmm. My mistake. I suppose I've been living in Java-land too long, and reading my viewpoint from there into other projects. Although, in my defence, PJE seems to say here: http://mail.python.org/pipermail/web-sig/2004-August/000668.html that once WSGI is a little more commonplace, it'll be easier to look at other places of duplicated functionality, and extract them out. Perhaps we'll have WSGI-Session, WSGI-Request/Response, WSGI-etc... It seems that we might already be moving that way, with stuff like WSGI Utils - http://www.owlfish.com/software/wsgiutils/ > I'd love to find out that I'm wrong, although I guess I would > find the WSGI idioms around accessing request headers/form-encoded > data and collecting response information a bit cumbersome compared > to the APIs provided by Webstack, Quixote, Webware, CherryPy, etc. Is there any commonality between those APIs? I'm sure people would be more than happy to conform to a standard API, especially if the source for the adapter was provided... It could possibly even make it into the WSGI PEP, if enough people found it useful. (Just reading through the PEP a little, I see the lines: > Why is this interface so low-level? I want feature X! (e.g. cookies, > sessions, persistence, ...) > > This isn't Yet Another Python Web Framework. It's just a way for > frameworks to talk to web servers, and vice versa. If you want these > features, you need to pick a web framework that provides the features > you want. And if that framework lets you create a WSGI application, > you should be able to run it in most WSGI-supporting servers. Also, > some WSGI servers may offer additional services via objects provided > in their environ dictionary; see the applicable server documentation > for details. (Of course, applications that use such extensions will > not be portable to other WSGI-based servers.) And what that says to me is: "If you want these features, don't use WSGI.", which is sort of strange, since WSGI could provide a reasonable API (or even just _an_ API) that applications could use to give themselves a modicum of framework-independance. Reading over this a little more, I think I might just have a completely wrong idea of what WSGI is, and where it's applicable, but having said that, given the plethora of frameworks out there, having something that webapp developers can code to to allow them to run unchanged on a few different frameworks (even if they have limited functionality) seems like a good idea. Is that the purpose behind WSGI, or is it the layer below, where frameworks talk to servers? Later, Blake. From paul at boddie.org.uk Fri Apr 15 22:35:14 2005 From: paul at boddie.org.uk (Paul Boddie) Date: Fri Apr 15 22:45:27 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <4260214B.50203@colorstudy.com> References: <200504150811.AA65798380@boddie.org.uk> <0968fdb63ef46c86e0fd023bfa457976@gmail.com> <4260214B.50203@colorstudy.com> Message-ID: <200504152235.14175.paul@boddie.org.uk> On Friday 15 April 2005 22:17, Ian Bicking wrote: > > In a way, I think this is up to the framework -- frameworks tend to have > opinions or policy on how files and code is layed out. Templates have > opinions on how site look is separate from the layout of individual > pages, and stuff like that. It's certainly more comforting to have conventions around where you put your files - I know that some of the small applications I've been writing end up with files in a fairly rigid directory hierarchy. But then we (meaning those of us who wish to somehow make frameworks and APIs "Pythonic" - how I hate to use that word) should at least try and make use of the way Python itself manages its resources. It's no good having packages in Python, for example, if you're not going to use them for organising resources. > Ideally, for a significant class of frameworks it would be nice if they > could all be interfaced in the same way. I.e., you pass in a namespace, > and maybe a "template space" as well, when templates look up up other > templates (as with ZPT macros, standard headers/footers in other other > languages, etc). There's small but subtle differences between > templates, but maybe those could be papered over. The complexities might arise in callbacks and what kind of data those callbacks operate on. I've found that various XSLT templating experiments have been made much more interesting with the use of extension functions. That said, I don't think anyone can really accuse XSLT of being framework-specific. > Then docs would be two-fold -- the framework would describe how it loads > and populates templates (generically) and templates would describe what > they do with the values given to them. I think that could be workable. Paul From jjinux at gmail.com Fri Apr 15 23:09:41 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri Apr 15 23:09:50 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <20050415201419.404A2C3@short.latte.ca> References: <8c3190f2657b445e5785a5bbc282459e@gmail.com> <20050415201419.404A2C3@short.latte.ca> Message-ID: > Reading over this a little more, I think I might just have a completely > wrong idea of what WSGI is, and where it's applicable, but having said > that, given the plethora of frameworks out there, having something that > webapp developers can code to to allow them to run unchanged on a few > different frameworks (even if they have limited functionality) seems > like a good idea. Is that the purpose behind WSGI, or is it the layer > below, where frameworks talk to servers? WSGI will not let you run code on different frameworks. It really is just a framework for frameworks to be able to share some code among themselves. Although I have the greatest respect for the developers of WSGI (I really do), I became a little disenchanted with WSGI when I discovered that it really doesn't help the developer who's coding all of his code to a particular framework. Hence, my efforts to address the current problem have been via tutorials, working with newbies on the Aquarium mailing list, and trying to achieve concensus among developers who have good standalone, reusable parts. If WSGI provides any benefit, it will simply be that it gets developers moving in the same direction. The WSGI developers understand that, and that would be a great thing! In the meantime, I'll continue to try to be the greatest by being the servant of the least. I.e., even if Aquarium isn't the most revolutionary thing since sliced bread, I do promise to bend over backwards helping newbies use it to meet their needs and fix it when it doesn't. I'm not touting my own horn so much as I am reminding you guys that it's the community that really matters. With a community where we each help each other, answer each other's questions, and share code whenever possible, it's a win no matter which framework or even which language gets the most mindshare. Happy Hacking, -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From mike_mp at zzzcomputing.com Fri Apr 15 23:37:39 2005 From: mike_mp at zzzcomputing.com (mike bayer) Date: Fri Apr 15 23:38:24 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <4260214B.50203@colorstudy.com> References: <200504150811.AA65798380@boddie.org.uk> <42600CB6.1040100@colorstudy.com> <0968fdb63ef46c86e0fd023bfa457976@gmail.com> <4260214B.50203@colorstudy.com> Message-ID: <5355.66.192.34.8.1113601059.squirrel@66.192.34.8> > Ideally, for a significant class of frameworks it would be nice if they > could all be interfaced in the same way. I.e., you pass in a namespace, > and maybe a "template space" as well, when templates look up up other > templates (as with ZPT macros, standard headers/footers in other other > languages, etc). There's small but subtle differences between > templates, but maybe those could be papered over. theres been a lot going on with Myghty with regards to the Resolver object, which is the object responsible for converting paths into sources of template component code, which usually come from template files stored in one or more root directories, but also can be defined within Python modules as well. of particular use is that you can pass in whatever kind of Resolver you want if you dont like the way it resolves. Now, since some people didnt like the way it resolved, *and* didnt want to write their own resolver, I will be adding in to the default resolver a full blown ruleset engine where you can drop in as many resolution rules as you like into your own custom rule lists. anyway, it would be straightforward to add new resolution rules that return sources to template objects from other platforms. the tricky part is that two templates from library A and library B both probably need to present a slightly adapted form of themselves to each other, like a multi-contextual object of some kind where each one can locate its familiar namespace bits. From paul at boddie.org.uk Fri Apr 15 23:30:23 2005 From: paul at boddie.org.uk (Paul Boddie) Date: Fri Apr 15 23:40:36 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <20050415201419.404A2C3@short.latte.ca> References: <20050415201419.404A2C3@short.latte.ca> Message-ID: <200504152330.23445.paul@boddie.org.uk> On Friday 15 April 2005 22:31, Blake Winton wrote: > > > I'd love to find out that I'm wrong, although I guess I would > > find the WSGI idioms around accessing request headers/form-encoded > > data and collecting response information a bit cumbersome compared > > to the APIs provided by Webstack, Quixote, Webware, CherryPy, etc. > > Is there any commonality between those APIs? Perhaps. WebStack originally took some inspiration from Webware and the Java Servlet API, but I think that the "vocabulary" is pretty standard, so it isn't necessarily that difficult to map WebStack API methods to framework methods. Quixote and CherryPy may have similar abstractions, but unlike most of the frameworks supported by WebStack, these frameworks impose various constraints (object publishing and template-orientation) that seemingly make it impractical to support them in an API at WebStack's level. Paul From ml at fgranger.com Fri Apr 15 23:42:53 2005 From: ml at fgranger.com (Fran=?ISO-8859-1?B?5w==?=ois Granger) Date: Fri Apr 15 23:42:51 2005 Subject: [Web-SIG] Revive HTMLGen Message-ID: I recently discovered HTMLGen. http://starship.python.net/crew/friedrich/HTMLgen/html/main.html It fits neatly with my idea of a Really Simple Web Service construction kit in Python. http://www.fgranger.com/cgi-bin/view/Development/ReallySimpleWebServicesPyth on I tried to contact the author and some contributors with no answers... I would like to revive it. From ianb at colorstudy.com Fri Apr 15 23:57:48 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sat Apr 16 00:00:52 2005 Subject: [Web-SIG] Revive HTMLGen In-Reply-To: References: Message-ID: <426038DC.90603@colorstudy.com> Fran?ois Granger wrote: > I recently discovered HTMLGen. > http://starship.python.net/crew/friedrich/HTMLgen/html/main.html > > It fits neatly with my idea of a Really Simple Web Service construction kit > in Python. > http://www.fgranger.com/cgi-bin/view/Development/ReallySimpleWebServicesPyth > on There's a bunch of alternatives that have been developed since then, most of which are a little simpler and more regular than HTMLGen. Nevow has stan, for instance, and I wrote a similar thing based on ElementTree: http://svn.colorstudy.com/trunk/Validator/validator/htmlgen.py There exist several other forms of the same thing out there too, and I've played around with it in a couple different forms, e.g.: http://svn.colorstudy.com/home/ianb/recipes/htmlstream.py http://svn.colorstudy.com/home/ianb/recipes/html.py -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From rtomayko at gmail.com Sat Apr 16 01:14:09 2005 From: rtomayko at gmail.com (Ryan Tomayko) Date: Sat Apr 16 01:14:15 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <200504152330.23445.paul@boddie.org.uk> References: <20050415201419.404A2C3@short.latte.ca> <200504152330.23445.paul@boddie.org.uk> Message-ID: <1ed1b536a385b52bdd6d7651a3b84aa7@gmail.com> On Apr 15, 2005, at 5:30 PM, Paul Boddie wrote: > On Friday 15 April 2005 22:31, Blake Winton wrote: >>> I'd love to find out that I'm wrong, although I guess I would >>> find the WSGI idioms around accessing request headers/form-encoded >>> data and collecting response information a bit cumbersome compared >>> to the APIs provided by Webstack, Quixote, Webware, CherryPy, etc. >> >> Is there any commonality between those APIs? > > Perhaps. WebStack originally took some inspiration from Webware and > the Java > Servlet API, but I think that the "vocabulary" is pretty standard, so > it > isn't necessarily that difficult to map WebStack API methods to > framework > methods. Quixote and CherryPy may have similar abstractions, but > unlike most > of the frameworks supported by WebStack, these frameworks impose > various > constraints (object publishing and template-orientation) that > seemingly make > it impractical to support them in an API at WebStack's level. Right - and that's fine. I'd rather see the frameworks follow a common set of "interfaces" for request/response/session/cookie/form objects then to have Webstack wrap the existing frameworks anyway. Object publishing, template orientation, form toolkits, template systems, etc. is where I see each framework distinguishing itself from the next. The basic set of objects that wrap the HTTP lifecycle are very similar and I don't think anyone is choosing a framework based on these components. You could split most Python based web frameworks into a couple layers I guess: 1. Gateway Interface and Middleware (WSGI) 2. HTTP Lifecycle Objects and APIs 3. Routing, Object Publishing, Servlet Mapping, Validation, Form Handling, etc. 4. Templating Layer #2 seems like another obvious place, like WSGI, where compatibility could be beneficial and not overly stifling to the individual framework. I think layer #3 is where things start getting really messy because the techniques vary from framework to framework. I realize I'm simplifying a bit - some pieces of functionality, like session management, blur between multiple layers and might be challenging to do well with additional restriction. Ryan From janssen at parc.com Sat Apr 16 01:16:01 2005 From: janssen at parc.com (Bill Janssen) Date: Sat Apr 16 01:16:27 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: Your message of "Fri, 15 Apr 2005 05:16:34 PDT." Message-ID: <05Apr15.161604pdt."58617"@synergy1.parc.xerox.com> Greg, > a buddy of mine down in the States was switching > to Ruby, after using Python for two years, because he and his > colleagues needed a lightweight, ready-out-of-the-box web app > framework We could discuss what lightweight means, I guess, but IMO there is a light-weight, pure-Python, ready-to-use Python web app framework, called Medusa. No other installs necessary -- you don't have to get some database running unless you really want to. Works remarkably well; I've written a few app servers in it already. Perhaps your buddy is actually switching for some other reason. Bill From david at sundayta.com Sat Apr 16 01:41:57 2005 From: david at sundayta.com (David Warnock) Date: Sat Apr 16 01:42:01 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <200504150811.AA65798380@boddie.org.uk> Message-ID: <42605145.3010603@sundayta.com> Greg, > > Paul Boddie wrote: > >> ...I firmly believe in "unbundling" templating languages from frameworks. > > > But doesn't that just make more work for the poor sods who are trying to > build things? After all, they have to rebundle them, don't they? Loose coupling between all the components of an application really helps in the long run. You gain hugely in flexibility and in longevity if you can swap in different components when required. Going for an all inclusive web framework can seem seductive, but for a large application it is dangerous. What happens when you meet the limits or when the support dies down. The ability to connect together best of breed is a much better long term solution. Dave From jjinux at gmail.com Sat Apr 16 02:04:58 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Sat Apr 16 02:05:02 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <3918380929196220453@unknownmsgid> References: <3918380929196220453@unknownmsgid> Message-ID: I'm sorry--I can't resist! :) On 4/15/05, Bill Janssen wrote: > We could discuss what lightweight means, I installed Aquarium and Glass (Aquarium's optional included Web server built on SimpleHTTPServer) on my PocketPC the other day :) I'd like to consider that lightweight! I was showing that to Guido yesterday, and he seemed to get a kick out of it. I must admit that it's much more of a testament to both Python and the guys who did the port than it is to Glass, but heh, whatever ;) > I guess, but IMO there is a > light-weight, pure-Python, ready-to-use Python web app framework, > called Medusa. Yep, written by Sam Rushing. He and I work together at IronPort. In a way, the successor of Medusa is via Stackless Python with coroutines. We're working out the details of open sourcing it. Glass can run on top of it, or it can use normal Python threads. > No other installs necessary -- you don't have to get > some database running unless you really want to. Works remarkably > well; I've written a few app servers in it already. Perhaps your > buddy is actually switching for some other reason. Ok, that's enough from me. Sorry for the interruption. Happy Hacking, -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From brendan.t.oconnor at gmail.com Sat Apr 16 02:54:24 2005 From: brendan.t.oconnor at gmail.com (Brendan O'Connor) Date: Sat Apr 16 02:54:31 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <425D190E.2020005@infrae.com> Message-ID: Amen. I don't mean to slam the work of any web framework developers, but you can't overestimate how much it helps just-beginning web developers to see a unified framework, or at least a good website that directs them to one particular way to do things. The python frameworks are all too small to support much of a user community. I personally had to pick PHP over Python a while back, partly for reasons of documentation and support -- if the community unified more around one or just a few frameworks, documentation and support would start to be solved better. -Brendan On 4/15/05, Greg Wilson wrote: > > It's been a week now since my "Just lost another one to Rails" post, > in which I said that a buddy of mine down in the States was switching > to Ruby, after using Python for two years, because he and his > colleagues needed a lightweight, ready-out-of-the-box web app > framework. Responses so far seem to fall into several camps: > > - "I agree completely, that's why I'm adding yet another framework to > the mix!" (I'm waiting for someone to stand up at PyCon and say, > "Web App People's Front? We're the People's Front of Web Apps!") > > - Sneering: "Bah---Rails is impure! Unclean! We must keep our Python > pure and elegant!" (Yeah... look how well that worked for Scheme. > In my experience, most programmers value usefulness over elegance.) > > - Whistling in the dark. For example, Ian Bicking said, "...diversity > isn't so bad if we can just make a compelling infrastructure > experience." I respectfully disagree: right now, the diversity in > this area is preventing any of the frameworks from becoming mature > enough to be credible among the "I need to get it done now" > developers I talk to. (Quick, how many copies of "Programming > WebWare/Twisted/CherryPy/whatever" or "The WW/T/CP/whatever > Cookbook" are on pre-order? Probably 3500 less than the equivalent > RonR books.) > > It also gives the impression of confusion and bickering, which is > lethal when you're trying to persuade someone in the commercial > world to adopt something that doesn't come with a 1-800 customer > support line. > > - Frank acknowledgment of RonR's strengths (e.g. Peter Hunt's very > welcome post --- Peter, I would have thanked you directly, but I > didn't have an email address). > > RonR is proof that new web app frameworks can displace existing tools > like PHP. It's also proof that the existence of a lightweight ready- > out-of-the-box don't-have-to-install-eleven-packages-to-make-it-work > yes-the-tutorials-are-up-to-date no-you-don't-have-to-write-lots-of- > idiosyncratic-XML-templates-or-configuration-files framework is > important enough that large numbers of programmers will choose (or > switch) their language on that basis alone. > > So, any bets we'll still be moaning about this after PyCon'06? > > Greg > > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/brendan.t.oconnor%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20050415/6daf83c5/attachment.htm From mike_mp at zzzcomputing.com Sat Apr 16 06:51:59 2005 From: mike_mp at zzzcomputing.com (michael bayer) Date: Sat Apr 16 06:52:16 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <425D190E.2020005@infrae.com> Message-ID: On Apr 15, 2005, at 8:54 PM, Brendan O'Connor wrote: > Amen.? I don't mean to slam the work of any web framework developers, > but you can't overestimate how much it helps just-beginning web > developers to see a unified framework, or at least a good website that > directs them to one particular way to do things.? this is what i dont understand; whats so hard about doing a little research ? why do people need to be told only one way of doing things ? python is not the the mcdonalds of computer languages...its sophisticated and thoughtful. If youre dealing with a thoughtful and sophisticated community, you need to approach it in a more thoughtful and sophisticated way. its reasonable to expect the community to work up some standards for interoperability....but less so to present a monolithic one-size-fits-all methodology to the masses so that nobody need be bothered with some thinking and decision making. that just dulls the community down to its most unsophisticated level and insures only the most mediocre outcomes. From ianb at colorstudy.com Sat Apr 16 22:12:35 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sat Apr 16 22:12:28 2005 Subject: [Web-SIG] WSGIKit and frameworks Message-ID: <426171B3.8080205@colorstudy.com> I've posted a couple things to my blog recently about WSGIKit: http://blog.ianbicking.org/what-is-wsgikit.html http://blog.ianbicking.org/what-can-wsgikit-do-for-you.html I'm really interested in getting other frameworks into WSGIKit, including providing whatever help I can, changing APIs as necessary, and providing access to the repository. I'm happy to hear any reactions or concerns. You can also talk more specifically about WSGIKit on webware-devel@lists.sf.net, see commits on webware-checkins@lists.sourceforge.net, and chat on #webware on freenode.net. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From floydophone at gmail.com Sun Apr 17 17:47:14 2005 From: floydophone at gmail.com (Peter Hunt) Date: Sun Apr 17 17:47:18 2005 Subject: [Web-SIG] Re: WSGIKit and frameworks Message-ID: <6654eac4050417084735291687@mail.gmail.com> Ian: does it support plugging in a custom WSGI application yet? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20050417/ccafc013/attachment.html From brendan.t.oconnor at gmail.com Sun Apr 17 19:18:21 2005 From: brendan.t.oconnor at gmail.com (Brendan O'Connor) Date: Sun Apr 17 19:18:25 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <425D190E.2020005@infrae.com> Message-ID: > > Amen. I don't mean to slam the work of any web framework developers, > > but you can't overestimate how much it helps just-beginning web > > developers to see a unified framework, or at least a good website that > > directs them to one particular way to do things. > > this is what i dont understand; whats so hard about doing a little > research ? why do people need to be told only one way of doing things > ? python is not the the mcdonalds of computer languages...its > sophisticated and thoughtful. If youre dealing with a thoughtful and > sophisticated community, you need to approach it in a more thoughtful > and sophisticated way. its reasonable to expect the community to > work up some standards for interoperability....but less so to present a > monolithic one-size-fits-all methodology to the masses so that nobody > need be bothered with some thinking and decision making. that just > dulls the community down to its most unsophisticated level and insures > only the most mediocre outcomes. Let me put it this way: I've spent weeks and weeks researching many many python frameworks, but never found one that seemed proven in terms of adoption by a large number of high-traffic sites. And since you can never really learn how good a framework is until you try it out, this information gathering had a tremendous cost! That cost slows down the entire community and keeps people out -- at some level, it's good just to have popularity among simple web developers, not just the people willing to hack around and create new frameworks. -Brendan > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/brendan.t.oconnor%40gmail.com > From ianb at colorstudy.com Sun Apr 17 21:46:51 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sun Apr 17 21:46:45 2005 Subject: [Web-SIG] Re: WSGIKit and frameworks In-Reply-To: <6654eac4050417084735291687@mail.gmail.com> References: <6654eac4050417084735291687@mail.gmail.com> Message-ID: <4262BD2B.6040104@colorstudy.com> Peter Hunt wrote: > Ian: does it support plugging in a custom WSGI application yet? Kind of. If you make a module with an application called "application", and put it in, say, /www/myapp/index.py, and set "webkit_dir='/www/myapp'" then it should work. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From paul at boddie.org.uk Sun Apr 17 23:26:26 2005 From: paul at boddie.org.uk (Paul Boddie) Date: Sun Apr 17 23:36:59 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <425D190E.2020005@infrae.com> Message-ID: <200504172326.26893.paul@boddie.org.uk> On Sunday 17 April 2005 19:18, Brendan O'Connor wrote: > > Let me put it this way: I've spent weeks and weeks researching many > many python frameworks, but never found one that seemed proven in > terms of adoption by a large number of high-traffic sites. Plone seems pretty widespread - one only needs to look at a lot of high profile open source sites (eg. Ubuntu Linux) to see that it's becoming very popular. However, much has already been said about whether it (and also Zope) is easy to develop for, and whether its focus is too narrow for any old Web application to be developed for it without overdue frustration. Paul From mike_mp at zzzcomputing.com Mon Apr 18 01:26:15 2005 From: mike_mp at zzzcomputing.com (michael bayer) Date: Mon Apr 18 01:26:21 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <425D190E.2020005@infrae.com> Message-ID: On Apr 17, 2005, at 1:18 PM, Brendan O'Connor wrote: > Let me put it this way: I've spent weeks and weeks researching many > many python frameworks, but never found one that seemed proven in > terms of adoption by a large number of high-traffic sites. one could say the same thing about Rails. > And since > you can never really learn how good a framework is until you try it > out, this information gathering had a tremendous cost! That cost > slows down the entire community and keeps people out -- at some level, > it's good just to have popularity among simple web developers, not > just the people willing to hack around and create new frameworks. for various reasons, some of which I think are historial, none of the frameworks have really had all the elements in place which would enable them to catch on to the degree necessary that there would be a substantial supporting community, established maturity of design, straightforward base of documentation, and ongoing progressive development, which is what the entering developer is looking for. But i dont think this is because there are too many frameworks. i think its just because none of them have caught on to a great enough degree, as of yet. i think no matter how many frameworks there are, if one of them is truly superior, easy to use, and can grow as much as the site can, it will become a winner all on its own (note I didnt say *the* winner). the current plethora presents no hindrance to that. From mike_mp at zzzcomputing.com Mon Apr 18 01:44:48 2005 From: mike_mp at zzzcomputing.com (michael bayer) Date: Mon Apr 18 01:44:53 2005 Subject: [Web-SIG] Re: WSGIKit and frameworks In-Reply-To: <4262BD2B.6040104@colorstudy.com> References: <6654eac4050417084735291687@mail.gmail.com> <4262BD2B.6040104@colorstudy.com> Message-ID: my particular confusion, and this may very well be because i am not digging in heartfully enough into the docs, is that the main example is very much bonded to using Webware and Zope templates, and since I am unfamiliar with those it is very difficult for me to see what parts of the example are specific to Webware/Zope and which are strictly part of WSGIKit (and therefore, the targets I would need to mold my own application to). but if I may ask one super-boneheaded question, im sure its entirely RTFM-worthy, when I look inside the example .py files underneath wsgikit/app_templates/webkit_zpt/, what am I seeing with variables like $app_name and ${app_name}, although underneath /examples/todo_sql/web/ they seem to be filled in. is that like "your code goes here" type of thing, or is there some script processing that ? On Apr 17, 2005, at 3:46 PM, Ian Bicking wrote: > Peter Hunt wrote: >> Ian: does it support plugging in a custom WSGI application yet? > > Kind of. If you make a module with an application called > "application", and put it in, say, /www/myapp/index.py, and set > "webkit_dir='/www/myapp'" then it should work. > > -- > Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/ > mike_mp%40zzzcomputing.com From ianb at colorstudy.com Mon Apr 18 04:55:05 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Mon Apr 18 04:54:56 2005 Subject: [Web-SIG] Re: WSGIKit and frameworks In-Reply-To: References: <6654eac4050417084735291687@mail.gmail.com> <4262BD2B.6040104@colorstudy.com> Message-ID: <42632189.7010408@colorstudy.com> michael bayer wrote: > my particular confusion, and this may very well be because i am not > digging in heartfully enough into the docs, is that the main example is > very much bonded to using Webware and Zope templates, and since I am > unfamiliar with those it is very difficult for me to see what parts of > the example are specific to Webware/Zope and which are strictly part of > WSGIKit (and therefore, the targets I would need to mold my own > application to). Well, it depends how you are coming at it. WSGIKit architecturally isn't tied to Webware or ZPT, but right now that's the only set of things that have been completely integrated. (Well, ZPT was integrated with Webware API, not with WSGIKit particularly.) So... if you just want to *use* WSGIKit, you have to use Webware right now. But if you are willing to hack about, the system is fairly flexible. The flexibility is not documented, but it might be easiest to respond to what you are specifically looking to do than go into the whole architecture, which I haven't figured out quite how to explain yet. > but if I may ask one super-boneheaded question, im sure its entirely > RTFM-worthy, when I look inside the example .py files underneath > wsgikit/app_templates/webkit_zpt/, what am I seeing with variables like > $app_name and ${app_name}, although underneath /examples/todo_sql/web/ > they seem to be filled in. is that like "your code goes here" type of > thing, or is there some script processing that ? When you run: app-setup --template=webkit_zpt create It fills in those files, so they are templates for the application. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From anthony at interlink.com.au Mon Apr 18 09:01:41 2005 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Apr 18 09:02:05 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <425D190E.2020005@infrae.com> References: <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> <425D190E.2020005@infrae.com> Message-ID: <200504181701.42294.anthony@interlink.com.au> On Wednesday 13 April 2005 23:05, Martijn Faassen wrote: > Buggy? I don't think ZCML is buggy. Where's that coming from? I wouldn't say ZCML is *buggy* as such, but it _is_ an utter pain in the arse to debug and get right. If you spell it wrong, you either get a broken web application with no useful traceback, or else a monstously hideous traceback that's almost entirely useless. The only way I found to do ZCML sanely was to copy an existing product's ZCML and modify it - this is nuts. > In the mean time, in the world of HTML templating, we see a lot more > agreement that sometimes a domain specific language is useful. People > generally don't want to be producing all their HTML from Python > functions. I've seen far less complaints about ZPT being cumbersome and > buggy. That's because when you make a mistake in ZPT, it's nearly always extremely obvious what the problem is. This is not the case with ZCML - well, unless this has changed dramatically since about the middle of last year. > That's not say ZCML as it stands doesn't have some didactic/learning > curve issues. But to reject this out of hand just like that is a bit too > easy, I think. Attempting to debug some random ZCML error was what drove me screaming away from Zope3 last year (having done a fair amount of work on and with it). I was spending far more time on debugging the "simple" task of configuration and hooking things together than I was on _actually_ _writing_ _code_. Screw that for a game of soldiers. I _like_ a lot of the stuff in Zope3. But ZCML was just too awful for me to continue working with it. It was too much like just changing stuff randomly and hitting it with a stick for my liking - there were far too many different XML tags with non-obvious names and what felt like a pile of dead chickens required to get it right. -- Anthony Baxter It's never too late to have a happy childhood. From renesd at gmail.com Mon Apr 18 10:41:55 2005 From: renesd at gmail.com (Rene Dudfield) Date: Mon Apr 18 10:41:57 2005 Subject: [Web-SIG] WSGIKit and frameworks In-Reply-To: <426171B3.8080205@colorstudy.com> References: <426171B3.8080205@colorstudy.com> Message-ID: <64ddb72c05041801411a55fd6@mail.gmail.com> Hi Ian, I'm interested in using any WSGI middleware, and servers you have in there. Do you have any documentation or examples on how to use those bits? WSGI middleware and servers should be quite reusable for other people wanting to pick bits from various frameworks/libraries. I really like Alan saddis page for his modular WSGI stuff with documentation. They are usually just one py file too!. http://www.saddi.com/software/py-lib/ On 4/17/05, Ian Bicking wrote: > I've posted a couple things to my blog recently about WSGIKit: > > http://blog.ianbicking.org/what-is-wsgikit.html > http://blog.ianbicking.org/what-can-wsgikit-do-for-you.html > > I'm really interested in getting other frameworks into WSGIKit, > including providing whatever help I can, changing APIs as necessary, and > providing access to the repository. I'm happy to hear any reactions or > concerns. You can also talk more specifically about WSGIKit on > webware-devel@lists.sf.net, see commits on > webware-checkins@lists.sourceforge.net, and chat on #webware on > freenode.net. > From amk at amk.ca Mon Apr 18 18:05:18 2005 From: amk at amk.ca (A.M. Kuchling) Date: Mon Apr 18 18:06:28 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <200504181701.42294.anthony@interlink.com.au> References: <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> <425D190E.2020005@infrae.com> <200504181701.42294.anthony@interlink.com.au> Message-ID: <20050418160518.GA1018@rogue.amk.ca> On Mon, Apr 18, 2005 at 05:01:41PM +1000, Anthony Baxter wrote: > If you spell it wrong, you either > get a broken web application with no useful traceback, or else a > monstously hideous traceback that's almost entirely useless. This is also part of why I've never been able to get anywhere with Twisted, and why I'm suspicious of systems that depend critically on interfaces, adaptors, and other such extensions to the object model. When they go wrong, diagnosing the problem is messy, and simple typos can get turned into huge debugging expeditions. --amk From james at pythonweb.org Mon Apr 18 20:40:21 2005 From: james at pythonweb.org (James Gardner) Date: Mon Apr 18 20:39:43 2005 Subject: [Web-SIG] Python Web Modules - Improved WSGI Support In-Reply-To: <64ddb72c05041801411a55fd6@mail.gmail.com> References: <426171B3.8080205@colorstudy.com> <64ddb72c05041801411a55fd6@mail.gmail.com> Message-ID: <4263FF15.5080402@pythonweb.org> Hi All, Whilst we are discussing WSGI implementations it might be worth mentioning that the Python Web Modules WSGI support has been vastly improved. There is some fairly comprehensive documentation and lots of examples (including a full application example which adds data to a database using forms and sign in) and a server on which to test them. Of particular note, I've implemented some of the suggestions made on this list: * Auth middleware can catch 403 status responses and automatically display a sign in form, handle sign in attempts, and redirect to the original page on successful sign in. * Error handling middleware can email error reports as well as display debug logs * Error document middleware can be used to catch unknown status codes and automatically display an appropriate error document either from a file, from a string or dynamically generated. Other improvements to the modules include a far better session API and an auth API that supports roles, access levels, groups, active/disabled, encryption, multiple applications and a much more sensible API. Cheers, James Website: http://www.pythonweb.org/projects/webmodules/ Documentation: http://www.pythonweb.org/projects/webmodules/doc/0.5.3/html_multipage/lib/wsgi.html Mailing list: http://lists.sourceforge.net/lists/listinfo/lemon-webmodules Download: http://www.pythonweb.org/projects/webmodules/release/0.5.3/ From janssen at parc.com Mon Apr 18 21:31:29 2005 From: janssen at parc.com (Bill Janssen) Date: Mon Apr 18 21:50:17 2005 Subject: [Web-SIG] WSGI for Medusa? Message-ID: <05Apr18.123130pdt."58617"@synergy1.parc.xerox.com> Anyone done a WSGI module for Medusa yet? Bill From faassen at infrae.com Tue Apr 19 17:29:31 2005 From: faassen at infrae.com (Martijn Faassen) Date: Tue Apr 19 17:26:02 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <200504181701.42294.anthony@interlink.com.au> References: <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> <425D190E.2020005@infrae.com> <200504181701.42294.anthony@interlink.com.au> Message-ID: <426523DB.20400@infrae.com> Anthony Baxter wrote: > On Wednesday 13 April 2005 23:05, Martijn Faassen wrote: > >>Buggy? I don't think ZCML is buggy. Where's that coming from? > > I wouldn't say ZCML is *buggy* as such, but it _is_ an utter pain > in the arse to debug and get right. If you spell it wrong, you either > get a broken web application with no useful traceback, or else a > monstously hideous traceback that's almost entirely useless. That traceback actually is useful fairly often, though it's indeed long and intimidating. > The only way I found to do ZCML sanely was to copy an existing > product's ZCML and modify it - this is nuts. I've had some horrifying experiences at Zope 3 development sprints while trying to do something new with Zope 3. I haven't had this in actual application development, and my experiences with Five. Adding a new page to an existing object, say, is not very complicated, and just one ZCML statement. I'm trying to say two things: * ZCML is imperfect, I agree. There are learning curve issues and sometimes it just gets hairy, though I suspect the hairiness is more about glueing up too many components which are spread out too much and are factored out too much... * I don't think ZCML, and the concept of ZCML as a domain specific language for glueing things together, are fundamentally wrong. >>In the mean time, in the world of HTML templating, we see a lot more >>agreement that sometimes a domain specific language is useful. People >>generally don't want to be producing all their HTML from Python >>functions. I've seen far less complaints about ZPT being cumbersome and >>buggy. > > That's because when you make a mistake in ZPT, it's nearly always > extremely obvious what the problem is. That is not completely my experience. I've had my share of long incomprehensible tracebacks going into the bowels of the ZPT engine. I think in part people feel more comfortable with templating engines than ZCML because they're far more familiar with the former. > This is not the case with ZCML - > well, unless this has changed dramatically since about the middle of > last year. I'm not sure when the ZCML rewrite landed, I think earlier than mid-last year though. >>That's not say ZCML as it stands doesn't have some didactic/learning >>curve issues. But to reject this out of hand just like that is a bit too >>easy, I think. > > Attempting to debug some random ZCML error was what drove me > screaming away from Zope3 last year (having done a fair amount of > work on and with it). I was spending far more time on debugging the > "simple" task of configuration and hooking things together than I was > on _actually_ _writing_ _code_. Screw that for a game of soldiers. Could you give me more details of what was going wrong that cost you so much time? Were you registering views and adapters for a simple web app, or doing something more complicated? I for one know that creating local services was an utter hell in Zope 3, but that's all gone now. > I _like_ a lot of the stuff in Zope3. But ZCML was just too awful for > me to continue working with it. It was too much like just changing > stuff randomly and hitting it with a stick for my liking This isn't my experience with ZCML when building a concrete application; I've been involved in a few Zope 2 applications that use Five by now, and it's mostly view registrations with a few edit forms thrown in here and there, and 1.5 adapters. I'd therefore really like to hear more about what you were doing and where this sense of frustration came from. > - there were > far too many different XML tags with non-obvious names and what > felt like a pile of dead chickens required to get it right. Okay, that's a complaint about the design of the language, and I agree that ZCML is far from optimal. I feel the tags are okay, but the amount of combinations of *attributes* to those tags sometimes get pretty ridiculous. Someone with a feel for language design could write a frontend on top of this that experiments with different ways of expressing the same thing. I don't think ZCML requires a lot of repeating of what you already said in the code, though. Another alternative I've been thinking about a bit since this discussion started here is to see how far you can push Python to get at least simple views going outside of ZCML, without losing the flexibility of being able to add a view to someone else's code witout changing it. I haven't gotten very far with this yet. Regards, Martijn From faassen at infrae.com Tue Apr 19 17:31:59 2005 From: faassen at infrae.com (Martijn Faassen) Date: Tue Apr 19 17:28:25 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <20050418160518.GA1018@rogue.amk.ca> References: <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> <425D190E.2020005@infrae.com> <200504181701.42294.anthony@interlink.com.au> <20050418160518.GA1018@rogue.amk.ca> Message-ID: <4265246F.8040700@infrae.com> A.M. Kuchling wrote: > On Mon, Apr 18, 2005 at 05:01:41PM +1000, Anthony Baxter wrote: > >>If you spell it wrong, you either >>get a broken web application with no useful traceback, or else a >>monstously hideous traceback that's almost entirely useless. > > > This is also part of why I've never been able to get anywhere with > Twisted, and why I'm suspicious of systems that depend critically on > interfaces, adaptors, and other such extensions to the object model. > When they go wrong, diagnosing the problem is messy, and simple typos > can get turned into huge debugging expeditions. I think this fairly accurately describes the risk of strongly decoupled systems like Zope 3 and Twisted. In the beginning of OO, people had this complaint about inheritance hierarchies too. And they were partially right, but not entirely. I believe adaptation is a useful tool, but like inheritance, one shouldn't overdo it and use with it with care. I can certainly say Zope 3 in some areas goes overboard with this. How would you deal with the flexibility requirements that these systems have, though? Regards, Martijn From faassen at infrae.com Tue Apr 19 17:53:05 2005 From: faassen at infrae.com (Martijn Faassen) Date: Tue Apr 19 17:49:38 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <200504150811.AA65798380@boddie.org.uk> References: <200504150811.AA65798380@boddie.org.uk> Message-ID: <42652961.70108@infrae.com> Hey, Paul Boddie wrote: [snip] > What I'm advocating is this: > > * That the community provides narrow/thin but *completely > separate* components/solutions which offer very well-defined > benefits - eg. Web APIs, templating systems, database access > layers. These things shouldn't be mixed up in the fundamental > system on the pretense of convenience. > * That documentation is produced to describe how one plugs these > things into each other and how one might go about integrating > other functionality into applications. > * That genuine solutions for certain styles of application may be > made, but not foisted on people from the lowest levels of any > given system. Some applications benefit from having .myapp on > the end of every Web resource, and by being able to write > "hello world" in a text file and have it pumped out dynamically > from the server; not all applications do, however. This sounds good. I think it would be good if Python web frameworks turned more into users of a cloud of focused, smaller, libraries and mini-frameworks. I've sighed a few times the last months when I ran into more and more Python-based schema and form frameworks. I developed Formulator for Zope 2 pretty early on, and was involved in 2002 in setting up Zope 3's schema framework, so I've contributed to the problem. In the Zope world there's also Archetypes, and I heard Archetypes is now working on a new generation which redesigns everything.. And I ran into Schevo and I saw formencode, which both have a history too, and then finally what made me sigh was running into a weblog post by Philip Eby on Spike. It all looks all very cool, but how many more do we need here? I believe that what we, framework developers, need is a bit more humility ("We're open, you just all plug into our great solution!" is not humble), and more active outreach to include pieces of other frameworks. While we also need to do a bit of work of opening up the neat bits of our frameworks to reuse by others, but that rather easily turns into the non-humble "Use mine!", so I think what we really need to do more of is the "Let's look for something to reuse!" variety of outreach than the "look, I made mine real great!" outreach (we'll do the latter automatically anyway :). I think Ian Bicking has been saying similar things. Zope has historically suffered from a sometimes massive lack of humility in this respect, though I think my community is not the only one which has been too interested in reinventing wheels. There have been various attempts in the past to open things up, and I've chipping away at Zope (3) a bit in my own way: http://faassen.n--tree.net/blog/view/weblog/2005/04/15/0 More excitingly, recently there's been an absolutely great iniative by others to integrate Zope 3 with Twisted using WSGI, and all kinds of coolness may develop from that eventually. Regards, Martijn From faassen at infrae.com Tue Apr 19 17:56:36 2005 From: faassen at infrae.com (Martijn Faassen) Date: Tue Apr 19 17:53:00 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <200504150811.AA65798380@boddie.org.uk> Message-ID: <42652A34.8090807@infrae.com> Greg Wilson wrote: > > Paul Boddie wrote: > >> ...I firmly believe in "unbundling" templating languages from frameworks. > > > But doesn't that just make more work for the poor sods who are trying to > build things? After all, they have to rebundle them, don't they? I think there's a difference between maintaining and developing something separately and not physically distributing something together. I agree that the Zope project, for instance, should be maintaining and distributing the ZODB separately from Zope, as they have been doing for some time now. But I certainly don't mind Zope shipping with right version of the ZODB included and having to only install Zope. If you really want to solve this, you'd end up with package management systems with dependency resolution. Most linux distributions have this. Let's not worry too much about physical distribution in this debate, more about how web frameworks should be organized and how they should relate to each other. Regards, Martijn From amk at amk.ca Tue Apr 19 17:58:01 2005 From: amk at amk.ca (A.M. Kuchling) Date: Tue Apr 19 17:59:15 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <4265246F.8040700@infrae.com> References: <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> <425D190E.2020005@infrae.com> <200504181701.42294.anthony@interlink.com.au> <20050418160518.GA1018@rogue.amk.ca> <4265246F.8040700@infrae.com> Message-ID: <20050419155801.GA10256@rogue.amk.ca> On Tue, Apr 19, 2005 at 05:31:59PM +0200, Martijn Faassen wrote: > How would you deal with the flexibility requirements that these systems > have, though? I have no idea what these requirements are. However, I do think that, if your implementation requires infrastructure that builds a whole new object model on top of Python, that may be a good clue that the implementation should be done in some other language than Python. --amk From faassen at infrae.com Tue Apr 19 18:20:41 2005 From: faassen at infrae.com (Martijn Faassen) Date: Tue Apr 19 18:17:05 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <20050419155801.GA10256@rogue.amk.ca> References: <05Apr12.122202pdt."58617"@synergy1.parc.xerox.com> <425D190E.2020005@infrae.com> <200504181701.42294.anthony@interlink.com.au> <20050418160518.GA1018@rogue.amk.ca> <4265246F.8040700@infrae.com> <20050419155801.GA10256@rogue.amk.ca> Message-ID: <42652FD9.2090909@infrae.com> A.M. Kuchling wrote: > On Tue, Apr 19, 2005 at 05:31:59PM +0200, Martijn Faassen wrote: > >>How would you deal with the flexibility requirements that these systems >>have, though? > > I have no idea what these requirements are. Someone builds some web app or framework/app, for instance, a CMS. Now someone else wants to extend or change the behavior of this app. For instance, someone wants to add a new page to document objects in a CMS, that contains some textual summary information (Word count, say). Other people do not want or need this functionality however, so they don't want to see this page. A more low level scenario could be modifications to the way URI traversal works (you hook in another adapter that knows how to look up a subobject), without having to change the core framework. > However, I do think that, > if your implementation requires infrastructure that builds a whole new > object model on top of Python, that may be a good clue that the > implementation should be done in some other language than Python. What new object model are you talking about? I mean, adapters and views (they're very similar, views provide user interfaces, and adapters developer interfaces). They're well established design patterns. Looking up a view or adapter in some registry is also fairly common. I see this scattered all across the Zope-based CMS I helped build, Silva, wherever we needed the flexibility (which was typically customer project driven, very real world all). I see them everywhere; views for content objects, metadata set mapping to content objects, XML importer/exporter to content objects, different behavior for binary content based on the mime type (such as showing different icons), and I'm sure there's more I'm forgetting.. All Zope 3 (which I know a lot better than Twisted) does with its adapter system is provide a uniform, flexible way to register these adapters and views and then later look them up, instead of having hand-written registries and different lookup APIs growing all across the system. Are you suggesting I switch to a language with, say, multi-methods instead, or is there another pattern to accomplish this kind of extensibility in Python that you favor? Regards, Martijn From ianb at colorstudy.com Tue Apr 19 18:36:48 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue Apr 19 18:39:03 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <42652961.70108@infrae.com> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> Message-ID: <426533A0.2090003@colorstudy.com> Martijn Faassen wrote: > I've sighed a few times the last months when I ran into more and more > Python-based schema and form frameworks. I developed Formulator for Zope > 2 pretty early on, and was involved in 2002 in setting up Zope 3's > schema framework, so I've contributed to the problem. In the Zope world > there's also Archetypes, and I heard Archetypes is now working on a new > generation which redesigns everything.. And I ran into Schevo and I saw > formencode, which both have a history too, and then finally what made me > sigh was running into a weblog post by Philip Eby on Spike. It all looks > all very cool, but how many more do we need here? > > I believe that what we, framework developers, need is a bit more > humility ("We're open, you just all plug into our great solution!" is > not humble), and more active outreach to include pieces of other > frameworks. While we also need to do a bit of work of opening up the > neat bits of our frameworks to reuse by others, but that rather easily > turns into the non-humble "Use mine!", so I think what we really need to > do more of is the "Let's look for something to reuse!" variety of > outreach than the "look, I made mine real great!" outreach (we'll do the > latter automatically anyway :). I think Ian Bicking has been saying > similar things. Well, since I show up in both paragraphs (having written FormEncode), and both sides of the issue, I guess I should reflect on why. The duplication of work did occur to me (and I was looking to give up at one time: http://blog.ianbicking.org/where-next-for-formencode.html). And I realized that I was spread too thin to give the necessary attention to all my still independently-developed projects. The resolution of such a situation can go a couple ways -- I abandon the project entirely, I somehow get other developers involved, or I try to fold the ideas into someone else's project. All three are very difficult, some in practical ways (how to get other people interested, how to find a compatible project) some emotionally (abandoning code, or the humility required to abandon code *and* work on someone else's project). I've gone through all three processes at times too -- FormEncode went dormant for a long time, then I kind of reminded myself of what it was useful for, then I looked around for other similar projects -- but I had no idea what to do once I found them. Say "hey, stop using what you are using, I got some better ideas"? I feel like an interloper. What I'm doing might apply to Archetypes or Zope schemas, but I'm not involved or invested *at all* with those projects, so my connection is tenuous. I feel like we have to meet in the middle somewhere, some neutral ground where people can be equally invested and involved... but I'm not sure how to do that, because what seems like a Big and Difficult Problem to me (form generation) may seem (at least at the moment) to be incidental to another project; useful, but not worth the coordination effort. Not to mention, what I think is important and distinguishing isn't what other people feel is important. (The other direction I considered was documentation: http://blog.ianbicking.org/a-theory-on-form-toolkits.html -- but is documenting a design aimed at decoupling that different than distributing the code that informed that design?) I need to figure this out, because WSGIKit is in the same position -- I'm trying to factor out something that I think deserves to be shared among projects, but how to I get other developers to adopt the project or participate in the development? Make It And They Will Come isn't that reliable. I think people are comfortable letting projects Compete In The Marketplace, i.e., put a bunch of stuff out there and see what comes out on top. But in reality nothing comes out on top, and reimplementation is common -- especially with something that tends to be mixed with an object model, like form generation. Everyone pretty much agrees that simple is better, but "simple" can be hard to define. Some projects -- Zope 3 in particular -- build up a bunch of new metaphors for programming, like adaptation, interfaces, utilities, and declarative glue. To a core Zope 3 developer these aren't complexity, they are fundamental ways of thinking about development. They are the prerequesites for understanding everything else. These prerequesites always exists -- I consider some level of OO understanding a foundation that I expect, and so I don't consider (many) OO techniques to be a sign of increased complexity. But for someone with only procedural experience this won't be the case. This is okay, because The World has agreed that OO is a fundamental concept. But there isn't a consensus on these new object models that people are working on, not even within the domain of the web. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From paul at boddie.org.uk Tue Apr 19 18:58:07 2005 From: paul at boddie.org.uk (Paul Boddie) Date: Tue Apr 19 18:58:10 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <42652961.70108@infrae.com> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> Message-ID: <200504191858.07820.paul@boddie.org.uk> On Tuesday 19 April 2005 17:53, Martijn Faassen wrote: > > This sounds good. I think it would be good if Python web frameworks > turned more into users of a cloud of focused, smaller, libraries and > mini-frameworks. That would be great! > I've sighed a few times the last months when I ran into more and more > Python-based schema and form frameworks. I developed Formulator for Zope > 2 pretty early on, and was involved in 2002 in setting up Zope 3's > schema framework, so I've contributed to the problem. In the Zope world > there's also Archetypes, and I heard Archetypes is now working on a new > generation which redesigns everything.. And I ran into Schevo and I saw > formencode, which both have a history too, and then finally what made me > sigh was running into a weblog post by Philip Eby on Spike. It all looks > all very cool, but how many more do we need here? For what it's worth, I've also developed some schema and form frameworks. ;-) I think the reason for the re-invention, also in the Web frameworks arena, is that people look at the design decisions and think, "That's horrible! I don't want to have to write that when doing that - why can't I just do this?" Not that certain aspects of, for example, Schevo don't look highly intriguing. > I believe that what we, framework developers, need is a bit more > humility ("We're open, you just all plug into our great solution!" is > not humble), and more active outreach to include pieces of other > frameworks. While we also need to do a bit of work of opening up the > neat bits of our frameworks to reuse by others, but that rather easily > turns into the non-humble "Use mine!", so I think what we really need to > do more of is the "Let's look for something to reuse!" variety of > outreach than the "look, I made mine real great!" outreach (we'll do the > latter automatically anyway :). I think Ian Bicking has been saying > similar things. The whole point of WebStack was to re-use as much as possible of the different frameworks. When I started it, I wanted to avoid dealing with the network plumbing and the tedious aspects of HTTP, although I'd have more confidence now to rewrite some of the higher-level things of the frameworks that I'm plugging into. Things like cgi.FieldStorage I wanted to remain opaque, but I'd feel more ready now to rewrite it if I felt that it would be a worthwhile activity. What I've sort of deduced from the discussions over the past few years is that people may want interoperability, but they want the magic cloud of interoperability to expose no API. WebStack gives you the possibility of deployment on eight frameworks but exposes an API which isn't cool enough, or something like that. But even WSGI exposes an API of sorts, and the only clear benefit it has over its chief inspiration, CGI, is that one isn't constantly reaching for the raw environment variables and I/O streams - it's either to its advantage that a well written CGI application could surely have been implemented on top of a similar layer years ago (since that would make such an application fairly open to porting now), or a statement that the road of consensus ends pretty quickly once you get out of town. > Zope has historically suffered from a sometimes massive lack of humility > in this respect, though I think my community is not the only one which > has been too interested in reinventing wheels. There have been various > attempts in the past to open things up, and I've chipping away at Zope > (3) a bit in my own way: > > http://faassen.n--tree.net/blog/view/weblog/2005/04/15/0 In my opinion, ClearSilver was one of the more under-rated template technologies. > More excitingly, recently there's been an absolutely great iniative by > others to integrate Zope 3 with Twisted using WSGI, and all kinds of > coolness may develop from that eventually. I honestly hope that the guts of Zope 3 don't perpetuate the nastiness found in Zope 2's HTTP objects. Paul From faassen at infrae.com Tue Apr 19 20:21:16 2005 From: faassen at infrae.com (Martijn Faassen) Date: Tue Apr 19 20:17:40 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <200504191858.07820.paul@boddie.org.uk> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> Message-ID: <42654C1C.50708@infrae.com> Paul Boddie wrote: > On Tuesday 19 April 2005 17:53, Martijn Faassen wrote: [snip] > For what it's worth, I've also developed some schema and form frameworks. ;-) > I think the reason for the re-invention, also in the Web frameworks arena, is > that people look at the design decisions and think, "That's horrible! I don't > want to have to write that when doing that - why can't I just do this?" Not > that certain aspects of, for example, Schevo don't look highly intriguing. Yes, I know that is the reason, it's why we keep reinventing wheels. And often we're right, it *could* be better, but: * you could choose to help to improve the existing system * you could stick with it anyway, dealing with the things that you don't like so much, as you're more efficient. I know it *feels* more efficient and productive and fun to build something fun, but I doubt it's actually so in many cases. And I'm just as guilty as everyone else. And I want my fun sometimes, and build new things. Integrating software can be fun too, though. I've found that more recently I've spent way more time integrating existing systems than building new ones: Zope 3 into Zope 2: Five, libxml2 properly into Python: lxml, and recently my Clarity experiments to integrate ClearSilver templates into Zope 3. Doing this integration work isn't particularly easy, but I can actually help make stuff other people seem to want to use too, as I'm not sitting alone in my little corner. This worked very well for Five and looks like it might work for lxml. Integration stuff is stuff too, after all. Anyway, that's just a rant, and I know people on this list already know this. :) [WebStack] [snip] > What I've sort of deduced from the discussions over the past few years is that > people may want interoperability, but they want the magic cloud of > interoperability to expose no API. WebStack gives you the possibility of > deployment on eight frameworks but exposes an API which isn't cool enough, or > something like that. I looked at WebStack before, it looks cool and is impressive, but I didn't have any need to deploy on eight frameworks, and working with such an abstraction would make some things harder, as you want to use the native facilities of the framework. WebStack abstracts away over a number of frameworks. This kind of stuff has worked for GUI toolkits, and you could say it works for HTML. It'd need some analysis to see whether this can work for web frameworks... Thinking "out loud". One reason portability for GUI toolkits is considered nice is because end users are using different OSes entirely, and you can't really change that easily. For web applications, you can more easily change what runs on the server, so that interoperability is less interesting, perhaps. Another benefit of WebStack could be that you could write a component and potentially reuse it over many frameworks in many applications. Reuse is intensely interesting for developers, and if a standard API encourages reuse of some form, then that's helpful. [snip] > In my opinion, ClearSilver was one of the more under-rated template > technologies. The data-driven concept is very intriguing; I sketched out some reasons in the weblog article. >>More excitingly, recently there's been an absolutely great iniative by >>others to integrate Zope 3 with Twisted using WSGI, and all kinds of >>coolness may develop from that eventually. > > I honestly hope that the guts of Zope 3 don't perpetuate the nastiness found > in Zope 2's HTTP objects. I am not sure, but I think they were rewritten. Regards, Martijn From jjinux at gmail.com Tue Apr 19 20:32:09 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Tue Apr 19 20:32:13 2005 Subject: [Web-SIG] ClearSilver In-Reply-To: <200504191858.07820.paul@boddie.org.uk> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> Message-ID: > In my opinion, ClearSilver was one of the more under-rated template > technologies. I think templating engines are really a matter of taste (or distaste ;) I worked with one of the developers of ClearSilver, Paul Clegg, but nonetheless my company decided not to use it in favor of Cheetah. These were the reasons I forwarded: 1. Serializing all of your data to some resource file format is silly and wasteful when Python is on both sides. It's better to just put all the data in a dict or something similarly simple and just pass the dict. ClearSilver makes more sense when your backend is written in C. 2. I know that some Web development teams have GUI guys that are completely in charge of the templates. That's cool. However, I've never worked at such a company. In every company I've worked for, the GUI guys give me a template and then I'm in charge of it. Since I'm in charge of it, I'm not afraid of using something like Cheetah. I *already know* Python, so learning Cheetah is trivial. Furthermore, I'm a "big boy" which means I'm capable of leaving the business logic mostly out of the view. On the other hand, I do use Cheetah to viciously cut down on HTML duplication. Just because it's HTML doesn't mean it's a good idea to duplicate stuff all over the place! Anyway, that's my $0.02. Happy Hacking, -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From ianb at colorstudy.com Tue Apr 19 22:31:40 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue Apr 19 22:33:40 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <200504191858.07820.paul@boddie.org.uk> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> Message-ID: <42656AAC.10205@colorstudy.com> Paul Boddie wrote: >>I've sighed a few times the last months when I ran into more and more >>Python-based schema and form frameworks. I developed Formulator for Zope >>2 pretty early on, and was involved in 2002 in setting up Zope 3's >>schema framework, so I've contributed to the problem. In the Zope world >>there's also Archetypes, and I heard Archetypes is now working on a new >>generation which redesigns everything.. And I ran into Schevo and I saw >>formencode, which both have a history too, and then finally what made me >>sigh was running into a weblog post by Philip Eby on Spike. It all looks >>all very cool, but how many more do we need here? > > > For what it's worth, I've also developed some schema and form frameworks. ;-) > I think the reason for the re-invention, also in the Web frameworks arena, is > that people look at the design decisions and think, "That's horrible! I don't > want to have to write that when doing that - why can't I just do this?" Not > that certain aspects of, for example, Schevo don't look highly intriguing. It occurs to me that in these cases, it would probably be more constructive to fork than to reimplement. I suppose that's the kind of environment that people who advocate distributed version control are thinking about. Or alternately repositories with relatively promiscuous commit access -- codespeak is doing this, for instance. Or, it would be convenient to set up subversion so that I could give commit access to limited locations (e.g., /home/username, or /project/experimental, or stuff like that) to self-registered users. I'm a little worried about security in that situation, but I also think that's probably the right methodology. > What I've sort of deduced from the discussions over the past few years is that > people may want interoperability, but they want the magic cloud of > interoperability to expose no API. WebStack gives you the possibility of > deployment on eight frameworks but exposes an API which isn't cool enough, or > something like that. Well, it's unclear to me if the deployment thing is very interesting, at least with the currently available options. No one (I'd hope) chooses Webware because the AppServer and installation process are so awesome. To make something like WebStack interesting, I think you have to allow for communication between the layers and a more intimately heterogeneous environment. But even then I'm not sure it's that useful. > But even WSGI exposes an API of sorts, and the only > clear benefit it has over its chief inspiration, CGI, is that one isn't > constantly reaching for the raw environment variables and I/O streams - it's > either to its advantage that a well written CGI application could surely have > been implemented on top of a similar layer years ago (since that would make > such an application fairly open to porting now), or a statement that the road > of consensus ends pretty quickly once you get out of town. If CGI didn't have such serious performance problems, I'd totally be using it for everything. So being associated with CGI isn't bad. It represents HTTP, and that's a good start. One thing that occurred to me while writing my WSGIKit presentation is that there's a parrallel when you avoid putting layout and design decisions in code; WSGI is a way of keeping API and aesthetic decisions out of the core infrastructure. That's not to say some more aesthetic conformity wouldn't be nice -- at least some conventions for the uncaring to follow -- but that's an entirely separate issue from WSGI. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From jjinux at gmail.com Thu Apr 21 02:53:55 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Thu Apr 21 02:53:59 2005 Subject: [Web-SIG] Aquarium release 2.2 In-Reply-To: References: Message-ID: A new version of Aquarium has been released. Here are the release notes and changelog: http://sourceforge.net/project/shownotes.php?release_id=322066 Many people are using Aquarium in new environments, and this release was focused on fixing bugs that popped up as a result. Bug fixes were necessary for: database connection pools, really big sessions, newbie HTML designers, reverse proxying, more professional error messages, the newest version of Cheetah, the newest version of MySQLdb, the VFAT filesystem, strange browsers, Windows users without a HOME, and inconvenient socket disconnects. Thanks go to the following engineers who contributed patches or submitted bugs for this release: Andrew Flury Anthony Eden Bob Van Zant Brandon Golm Jeffrey Wescott Krishna Srinivasan Kyle Vanderbeek Oleksandr Sukholeyster Paul Lorence Robert Leftwich From ianb at colorstudy.com Fri Apr 22 10:25:21 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 22 10:25:06 2005 Subject: [Web-SIG] WSGIKit renamed as Python Paste Message-ID: <4268B4F1.30504@colorstudy.com> I've renamed WSGIKit as "Python Paste". Credit for the name goes to Chad Walstrom. New links: * Website: http://pythonpaste.org * Mailing list: paste-users@pythonpaste.org -- http://pythonpaste.org/mailman/listinfo/paste-users * Trac bugtracker: http://pythonpaste.org/trac/ * Subversion repository: http://svn.pythonpaste.org/Paste/trunk/ * New package name: paste -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From jjinux at gmail.com Tue Apr 26 03:11:46 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Tue Apr 26 03:11:49 2005 Subject: [Web-SIG] JavaScript escape function Message-ID: Hey guys, I need a JavaScript escape function. Let's say I have a variable, and I'm generating some JavaScript from Cheetah: var s = "$s"; I need to make $s safe: var s = "$javascript_safe($s)"; Has anyone coded this yet? Will the same function work for both single and double quoted strings? Can I steal some code? Thanks :-D -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From ianb at colorstudy.com Tue Apr 26 03:18:33 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue Apr 26 03:18:13 2005 Subject: [Web-SIG] JavaScript escape function In-Reply-To: References: Message-ID: <426D96E9.9090706@colorstudy.com> Shannon -jj Behrens wrote: > Hey guys, > > I need a JavaScript escape function. Let's say I have a variable, and > I'm generating some JavaScript from Cheetah: > > var s = "$s"; > > I need to make $s safe: > > var s = "$javascript_safe($s)"; > > Has anyone coded this yet? Will the same function work for both > single and double quoted strings? Can I steal some code? FWIW, Python string literals seem to match Javascript's perfectly (at least the strings repr() produces -- not triple-quoted strings or anything fancy). So you'd do "var s = $repr(str(s))". -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From exarkun at divmod.com Tue Apr 26 03:19:27 2005 From: exarkun at divmod.com (Jp Calderone) Date: Tue Apr 26 03:19:30 2005 Subject: [Web-SIG] JavaScript escape function In-Reply-To: Message-ID: <20050426011927.15422.424230129.divmod.quotient.372@ohm> On Mon, 25 Apr 2005 18:11:46 -0700, Shannon -jj Behrens wrote: >Hey guys, > >I need a JavaScript escape function. Let's say I have a variable, and >I'm generating some JavaScript from Cheetah: > > var s = "$s"; > >I need to make $s safe: > > var s = "$javascript_safe($s)"; > >Has anyone coded this yet? Will the same function work for both >single and double quoted strings? Can I steal some code? > >Thanks :-D >-jj > Nevow has liveevil.jquote(), which I believe does what you want. It is strikingly simple and straightforward: def jquote(jscript): return jscript.replace('\\', '\\\\' ).replace("'", "\\'" ).replace('\n', '\\n') I wonder if it isn't buggy somehow :) I don't know javascript well enough myself to say either way. Jp From dp at ulaluma.com Tue Apr 26 04:26:39 2005 From: dp at ulaluma.com (Donovan Preston) Date: Tue Apr 26 04:26:44 2005 Subject: [Web-SIG] JavaScript escape function In-Reply-To: <20050426011927.15422.424230129.divmod.quotient.372@ohm> References: <20050426011927.15422.424230129.divmod.quotient.372@ohm> Message-ID: On Apr 26, 2005, at 1:19 AM, Jp Calderone wrote: > > Nevow has liveevil.jquote(), which I believe does what you want. > It is strikingly simple and straightforward: > > def jquote(jscript): > return jscript.replace('\\', '\\\\' > ).replace("'", "\\'" > ).replace('\n', '\\n') > > I wonder if it isn't buggy somehow :) I don't know javascript > well enough myself to say either way. This is for explicitly quoting strings for inclusion in a javascript function call which is being built, which is why it is quoting ' For example: nevow_clientToServerEvent('identifier', 'hello(\'world\')') The second argument would have been a python string which looked like this, which was then run through jquote before being used to build the nevow_clientToServerEvent call: "hello('world')" To apply this to jj's original example, assigning a javascript string to a javascript variable, use single quotes around the call to jquote, or switch jquote to quote " instead of ' var s = '$jquote($s)' Leaving it up to the developer to ensure every potentially unsafe python string is quoted properly always makes me nervous. Nevow has enough information to automatically safely quote strings inserted in html, html attribute, and URL contexts, but isn't currently smart enough to know much about javascript and javascript string quoting contexts. livepage will try to quote things properly for you if you are using livepage.handler (which uses jquote as shown above) but it's pretty easy to fool, because nevow doesn't yet know enough about JavaScript to really be safe. I hope to add some additional intelligence soon to cover this, and perhaps also to cover CSS contexts. dp From ianb at colorstudy.com Tue Apr 26 04:34:18 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue Apr 26 04:33:55 2005 Subject: [Web-SIG] JavaScript escape function In-Reply-To: References: <20050426011927.15422.424230129.divmod.quotient.372@ohm> Message-ID: <426DA8AA.7040208@colorstudy.com> Donovan Preston wrote: > var s = '$jquote($s)' > > Leaving it up to the developer to ensure every potentially unsafe > python string is quoted properly always makes me nervous. Nevow has > enough information to automatically safely quote strings inserted in > html, html attribute, and URL contexts, but isn't currently smart > enough to know much about javascript and javascript string quoting > contexts. livepage will try to quote things properly for you if you are > using livepage.handler (which uses jquote as shown above) but it's > pretty easy to fool, because nevow doesn't yet know enough about > JavaScript to really be safe. I hope to add some additional > intelligence soon to cover this, and perhaps also to cover CSS contexts. This is why I strongly prefer keeping explicit quotes out of the substitution, i.e., "var s = $repr(str(s))" (or jquote or whatever), because it's less likely to lead to errors. If you do "var s = $s" it'll just break (syntactically invalid). But if you do "var s = '$s'" it'll work most of the time. This is how the DB-API (and database quoting generally) works. And not how PHP quoting is usually done, and we know how that ends up ;) -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From dp at ulaluma.com Tue Apr 26 05:32:41 2005 From: dp at ulaluma.com (Donovan Preston) Date: Tue Apr 26 05:32:47 2005 Subject: [Web-SIG] JavaScript escape function In-Reply-To: <426DA8AA.7040208@colorstudy.com> References: <20050426011927.15422.424230129.divmod.quotient.372@ohm> <426DA8AA.7040208@colorstudy.com> Message-ID: On Apr 25, 2005, at 7:34 PM, Ian Bicking wrote: > Donovan Preston wrote: > >> var s = '$jquote($s)' >> Leaving it up to the developer to ensure every potentially unsafe >> python string is quoted properly always makes me nervous. Nevow >> has enough information to automatically safely quote strings >> inserted in html, html attribute, and URL contexts, but isn't >> currently smart enough to know much about javascript and >> javascript string quoting contexts. livepage will try to quote >> things properly for you if you are using livepage.handler (which >> uses jquote as shown above) but it's pretty easy to fool, because >> nevow doesn't yet know enough about JavaScript to really be safe. >> I hope to add some additional intelligence soon to cover this, >> and perhaps also to cover CSS contexts. >> > > This is why I strongly prefer keeping explicit quotes out of the > substitution, i.e., "var s = $repr(str(s))" I agree. It looks like $repr(str(s)) should do the Right Thing all the time here. > (or jquote or whatever) jquote was never meant to be used explicitly. It was always meant to be used by handler, which knows exactly what the current quote situation is, like this: @livepage.handler(somePotentiallyUnsafeThing) def foo(client, theUnsafeThingHandledSafely): print "Your unsafe thing got quoted properly: ", theUnsafeThingHandledSafely > , because it's less likely to lead to errors. If you do "var s = > $s" it'll just break (syntactically invalid). But if you do "var s > = '$s'" it'll work most of the time. Except when s contains ', \\, or \n. > This is how the DB-API (and database quoting generally) works. And > not how PHP quoting is usually done, and we know how that ends up ;) I'm trying to show my wife how to get some simple things done with PHP, mostly because there are huge reference books she can read and follow. (It's unfortunate that there are no nice Python solutions I can teach her, including my own, but we'll fix that, right?) In the example I'm following to construct some SQL, they do: $bar = 'some string' $baz = 42 $query = "INSERT INTO foo VALUES ('$bar', $baz)" Are you telling me if $bar contains ' then all hell will break loose? Donovan From renesd at gmail.com Tue Apr 26 06:23:32 2005 From: renesd at gmail.com (Rene Dudfield) Date: Tue Apr 26 06:23:35 2005 Subject: [Web-SIG] JavaScript escape function In-Reply-To: References: Message-ID: <64ddb72c050425212341398d8e@mail.gmail.com> repr works quite well, and I've used it with lots of different javascript... However I'm not completely confident in it as it fails for unicode. eg. >>> a = u'a=\'asdfd\';\n\r\n' >>> a u"a='asdfd';\n\r\n" >>> print repr(a) u"a='asdfd';\n\r\n" This is an invalid js string. eek. So first we need to translate it to the correct charset encoding for the document. However js and python encoding of unicode in strings is different too. Here is one below which kind of works for js unicode stuff. But not correctly, and the output isn't the best(too many backslashes), and it isn't the fastest. def encode_js(text, charset = None): if type(text)==type(u""): text= text.encode([charset, 'utf-8'][charset==None]) text = text.replace("\r", "\\r").replace("\n", "\\n") text = text.replace('"', '\\"').replace("'", "\\'") return "'" + text + "'" This would be a good function to get right. We need the specifications, as well as unittests first :) On 4/26/05, Shannon -jj Behrens wrote: > Hey guys, > > I need a JavaScript escape function. Let's say I have a variable, and > I'm generating some JavaScript from Cheetah: > > var s = "$s"; > > I need to make $s safe: > > var s = "$javascript_safe($s)"; > > Has anyone coded this yet? Will the same function work for both > single and double quoted strings? Can I steal some code? > > Thanks :-D > -jj > > -- > I have decided to switch to Gmail, but messages to my Yahoo account will > still get through. > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/renesd%40gmail.com > From dp at ulaluma.com Tue Apr 26 06:41:27 2005 From: dp at ulaluma.com (Donovan Preston) Date: Tue Apr 26 06:41:33 2005 Subject: [Web-SIG] JavaScript escape function In-Reply-To: <64ddb72c050425212341398d8e@mail.gmail.com> References: <64ddb72c050425212341398d8e@mail.gmail.com> Message-ID: On Apr 25, 2005, at 9:23 PM, Rene Dudfield wrote: > repr works quite well, and I've used it with lots of different > javascript... However I'm not completely confident in it as it fails > for unicode. Ah, unicode, yes. Nevow handles unicode at a different abstraction level, in the main flatten loop, nevow.flat.ten.iterflatten. It encodes to utf-8 by default. Nevow's flattening process does a very good job of ensuring the right quoting takes place in the right context. With a little bit of tweaking, perhaps this core conversion from python data structures to correctly encoded bytes for transmission over the wire could be broken out from Nevow so other projects could benefit from it. Quoting is so difficult to get right that if many people reimplement quoting, some of the implementations are sure to be wrong simply because there are so many corner cases. It seems like this might be one area where it would be both beneficial and possible to share code between Python web frameworks. dp -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20050425/a45501bb/attachment.html From jjinux at gmail.com Tue Apr 26 08:02:29 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Tue Apr 26 08:02:31 2005 Subject: [Web-SIG] JavaScript escape function In-Reply-To: References: <20050426011927.15422.424230129.divmod.quotient.372@ohm> <426DA8AA.7040208@colorstudy.com> Message-ID: > I'm trying to show my wife how to get some simple things done with > PHP, mostly because there are huge reference books she can read and > follow. I suggest "Core PHP", but then that's because I was technical editor ;) > (It's unfortunate that there are no nice Python solutions I > can teach her, including my own, but we'll fix that, right?) In the > example I'm following to construct some SQL, they do: > > $bar = 'some string' > $baz = 42 > $query = "INSERT INTO foo VALUES ('$bar', $baz)" > > Are you telling me if $bar contains ' then all hell will break loose? Yep. Back when I was doing PHP programming, for some reason we always created entire queries in this way (instead of using some argument mechanism), but we made sure to do quoting explicitly. I've seen "all hell break loose" when I forgot to do an escape. Best Regards, -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From jjinux at gmail.com Tue Apr 26 08:22:57 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Tue Apr 26 08:23:00 2005 Subject: [Web-SIG] JavaScript escape function In-Reply-To: References: <64ddb72c050425212341398d8e@mail.gmail.com> Message-ID: > Quoting is so difficult to get right that if many people reimplement > quoting, some of the implementations are sure to be wrong simply because > there are so many corner cases. It seems like this might be one area where > it would be both beneficial and possible to share code between Python web > frameworks. Well, I'm sure someone will come along and out do me, but how about the following: def javaScriptEscape(s): """Prepare ``s`` for use in a JavaScript quoted string. Both ``"`` and ``'`` are escaped, so you can use the return value in either single or double quotes. """ if not isinstance(s, unicode): s = str(s) # Never call str on unicode. s = "\"'" + s # Force repr to use single quotes. s = `s` start, end = 4, -1 # Strip outer quotes and added quotes. if s.startswith("u"): # JS strings are implicitly unicode. start += 1 s = s[start:end] s = s.replace('"', '\\"') # Escape double quotes too. return s def javaScriptQuote(s): """Escape ``s`` and wrap it in single quotes.""" return "'%s'" % javaScriptEscape(s) # Do some testing. if __name__ == '__main__': for (k, v) in [("", ""), ("a", "a"), ("\t", "\\t"), ("\n", "\\n"), ("'", "\\'"), ('"', '\\"'), ("\377", "\\xff"), ("\xff", "\\xff"), (u"\u1234", "\\u1234")]: escaped = javaScriptEscape(k) if escaped != v: raise AssertionError( "javaScriptEscape(%s) led to %s instead of %s" % (`k`, `escaped`, `v`)) assert javaScriptQuote("foo\n") == "'foo\\n'" Note, I purposely chose to have two separate functions, because I sometimes want one, and I sometimes want the other. If anyone wants this code, please consider it in the public domain. On the other hand, I wouldn't mind being included in your CONTRIB. Thanks! Best Regards, -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From mmzeeman at xs4all.nl Tue Apr 26 08:32:02 2005 From: mmzeeman at xs4all.nl (Maas-Maarten Zeeman) Date: Tue Apr 26 08:32:03 2005 Subject: [Web-SIG] Status of PEP 268 Message-ID: <426DE062.8000107@xs4all.nl> Hello, It looks like PEP 268 Extended HTTP functionality and WebDAV (http://www.python.org/peps/pep-0268.html) is dead. Personally I have run into a couple of situations where it is just not possible to use urllib or urllib2 (e.g adding more efficient Digest authentication, Negotiate authentication, gzip encoding, etc). The idea explained in the PEP looks good. It looks like no one is interested in completing/implementing/finishing this PEP. Is this the case? Regards, Maas From p.f.moore at gmail.com Tue Apr 26 10:29:46 2005 From: p.f.moore at gmail.com (Paul Moore) Date: Tue Apr 26 10:29:49 2005 Subject: [Web-SIG] JavaScript escape function In-Reply-To: References: <20050426011927.15422.424230129.divmod.quotient.372@ohm> <426DA8AA.7040208@colorstudy.com> Message-ID: <79990c6b05042601295266048f@mail.gmail.com> On 4/26/05, Donovan Preston wrote: > I'm trying to show my wife how to get some simple things done with > PHP, mostly because there are huge reference books she can read and > follow. (It's unfortunate that there are no nice Python solutions I > can teach her, including my own, but we'll fix that, right?) In the > example I'm following to construct some SQL, they do: > > $bar = 'some string' > $baz = 42 > $query = "INSERT INTO foo VALUES ('$bar', $baz)" > > Are you telling me if $bar contains ' then all hell will break loose? Very definitely. With something like $bar = 'joe\'s string' the "hell" is a relatively benign syntax error in the SQL. However, if I had a bit of time, I could construct a value for $bar which could do all sorts of exciting things to your data. This is what people are talking about when they refer to "SQL Injection" attacks. Google for "SQL Injection" if you want to know more. Or for a web page which walks you through how to execute a SQL injection attack on a website, look at http://www.unixwiz.net/techtips/sql-injection.html - great reading if you want to increase your level of paranoia :-) Paul. From paul at boddie.org.uk Tue Apr 26 17:16:21 2005 From: paul at boddie.org.uk (Paul Boddie) Date: Tue Apr 26 17:16:35 2005 Subject: [Web-SIG] Status of PEP 268 In-Reply-To: <426DE062.8000107@xs4all.nl> References: <426DE062.8000107@xs4all.nl> Message-ID: <200504261716.21862.paul@boddie.org.uk> On Tuesday 26 April 2005 08:32, Maas-Maarten Zeeman wrote: > Hello, > > It looks like PEP 268 Extended HTTP functionality and WebDAV > (http://www.python.org/peps/pep-0268.html) is dead. Personally I have > run into a couple of situations where it is just not possible to use > urllib or urllib2 (e.g adding more efficient Digest authentication, > Negotiate authentication, gzip encoding, etc). The idea explained in the > PEP looks good. Regarding the WebDAV stuff, I note that the davlib module was supposedly developed - I suppose it provided the material for that part of the PEP. I'm tempted to look into this a bit further, having recently experimented with making WebDAV services. > It looks like no one is interested in completing/implementing/finishing > this PEP. Is this the case? I think that the urllib, urllib2, httplib modules (and so on) really need reviewing and a more coherent package structure introduced, possibly with more accessible classes for cases where one needs to change the default behaviour, especially in the simple cases. Perhaps there'll be a sprint at EuroPython around such things... Paul From gvwilson at cs.utoronto.ca Tue Apr 26 19:29:32 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Tue Apr 26 19:37:11 2005 Subject: [Web-SIG] ann: Data Crunching Message-ID: *ahem* Readers of this newsgroup might be interested in a new book on data crunching, which is available from Amazon: http://www.amazon.com/exec/obidos/ASIN/0974514071 or directly from the Pragmatic Programmers: http://www.pragmaticprogrammer.com/titles/gwd/index.html The book covers basic text processing, regular expressions, XML manipulation, binary data handling, and the 10% of relational databases that every programmer should know. Most of the examples are in Python (though Unix command line tools, XSL, and SQL are in there as well). Hope you enjoy it, Greg Wilson (its grinning author) From ianb at colorstudy.com Wed Apr 27 00:43:00 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed Apr 27 00:42:36 2005 Subject: [Web-SIG] WSGI URLOpener Message-ID: <426EC3F4.3010001@colorstudy.com> Just an idea I'll put out, in case anyone finds it interesting... A WSGI opener for urllib2 would be interesting. I imagine using it like: import urllib2 import wsgiurl myapp = construct_app() wsgiurl.register_app('myapp', myapp) f = urllib2.urlopen('wsgi:myapp/login') and so on. Then things like PBP and Twill could, I think, be run on top of an in-process WSGI application. Though I suspect that some of these might like like the wsgi: scheme...? If so, an option like: wsgiurl.register_app('myapp', myapp, scheme='http') f = urllib2.urlopen('http://myapp/login') Would be useful. This would probably require some monkeypatching of urllib2 to intercept calls to some domains. Either way, this makes functional and acceptance testing easier and faster to perform (since there's no server setup), and means that the same tools can easily be used for both in-process and remote testing. Anyway, I think it would be neat, and maybe one of you guys will think it's a neat idea and implement it...? ;) -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From gvwilson at cs.utoronto.ca Wed Apr 27 16:16:39 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Wed Apr 27 16:21:16 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <42654C1C.50708@infrae.com> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> Message-ID: Well, here we all are, two weeks later, with no sign of convergence on a solution. The mailing list set up in the wake of Michelle Levesque's "PyWebOff" talk at PyCon seems to have gone quiet; meanwhile, even Nathan Torkington (of Perl Cookbook fame) is raving about Rails: Ruby on Rails is astounding. Using it is like watching a kung-fu movie, where a dozen bad-ass frameworks prepare to beat up the little newcomer only to be handed their asses in a variety of imaginative ways. (from http://radar.oreilly.com/archives/2005/04/on_conferences.html) So, as far as I can tell, the most credible Ruby-like initiative going right now in the Python world is Subway: http://subway.python-hosting.com/ which combines SQLObject, Cheetah's templating, CherryPy, and a form encoding package. Anyone have anything better to offer? Thanks, Greg From ianb at colorstudy.com Wed Apr 27 18:04:44 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed Apr 27 18:07:10 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> Message-ID: <426FB81C.5060508@colorstudy.com> Greg Wilson wrote: > Well, here we all are, two weeks later, with no sign of convergence on a > solution. Well, Peter Hunt has code to run Subway on Paste, and is planning for the next release to be based on Paste. That will also make CherryPy available in Paste as a side effect. Mike Orr has been doing some work on running Quixote on Paste, though I don't know exactly what his current thoughts are; there's been interest from several Quixote developers, so something is bound to catch. And I'm pretty sure I convinced Adrian Holovaty to base the initial open source release of his company's web framework/CMS on Paste. Sure, I'd like to see even more people moving frameworks onto Paste, but I'm also pleased with the response so far, and hopeful we'll see other frameworks come along in time. I'm always ready to field questions about the details. Most of my own Paste-related work in the last week or two has been in setting up project infrastructure -- bug trackers, mailing lists, etc -- but most of what I plan to do in that area is done and I hope to move back to more significant code improvements. And I'd be just as happy to work on code improvements driven by the needs of other frameworks. And I'm always happy to include new developers, even if Paste is an aside to your primary efforts (like your own app or framework). > The mailing list set up in the wake of Michelle Levesque's > "PyWebOff" talk at PyCon seems to have gone quiet; I wasn't even aware of such a list...? Mailing lists go quiet if they aren't based on something concrete. Heck, Web-SIG regularly goes quiet for months, but I still think it's useful. > meanwhile, even > Nathan Torkington (of Perl Cookbook fame) is raving about Rails: > > Ruby on Rails is astounding. Using it is like watching a kung-fu > movie, where a dozen bad-ass frameworks prepare to beat up the > little newcomer only to be handed their asses in a variety of > imaginative ways. > (from http://radar.oreilly.com/archives/2005/04/on_conferences.html) Well then go and make some cool web applications and then come back and give feedback to the framework authors about what they could do to make that more pleasing. Rinse and repeat, and talk about it in public all along the way, and we'll see what comes out of it. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From faassen at infrae.com Wed Apr 27 19:14:03 2005 From: faassen at infrae.com (Martijn Faassen) Date: Wed Apr 27 19:09:26 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> Message-ID: <426FC85B.5010309@infrae.com> Greg Wilson wrote: > Well, here we all are, two weeks later, with no sign of convergence on a > solution. I'm not sure whether a 'solution' is possible. Nathan Torkington's rave is typical hype in my mind. Hyping some other system might help, but I don't think the web-sig can organize such a thing. Concerning actual technical convergence from a development point of view, besides the Paste work that Ian Bicking lists in his mail, Zope 3 and Twisted are being integrated using WSGI. On my personal agenda is to check out Ian's formencode and see how we can use it for Zope 3; the little I read about the philosophy (especially the bit why auto-generating forms, while seemingly attractive, has problems) struck me very positively. What kind of solution are you looking for? One can hardly expect people to drop the frameworks they've invested a lot in and all converge upon another one. You can't even expect Python programmers to actually stop creating new frameworks. I think evolution can work, and the trend is going in the right direction -- the Zope 3/Twisted development plus the Paste and Subway developments means there's movement in the direction of integration of existing code, instead of reinventing the wheel all the time. We need to keep this up and expand it further. Regards, Martijn From gvwilson at cs.utoronto.ca Wed Apr 27 20:26:09 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Wed Apr 27 20:29:32 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> Message-ID: > > Greg Wilson wrote: > > Well, here we all are, two weeks later... > Ian Bicking: > [Subway on Paste] > [Quixote on Paste] > [Adrian Holovaty's stuff on Paste] There's an old Chinese proverb (well, I was told it's Chinese): "If a hundred sages each tell you something different, the odds are that they're all wrong." I think WSGIKit/Paste is a great idea, but as long as a dozen people keep saying, "Use mine! No, use mine!", the average "need to get it done for Tuesday" web developer isn't going to touch any of them. > Martijn Faassen: > One can hardly expect people to drop the frameworks they've invested a > lot in and all converge upon another one. You can't even expect Python > programmers to actually stop creating new frameworks. Expect? No. Ask? Yes. Believe that if they don't, they'll be wondering in five years why Python has become the next Tcl, instead of the next Perl? Absolutely. Thanks, Greg From titus at caltech.edu Wed Apr 27 20:44:18 2005 From: titus at caltech.edu (Titus Brown) Date: Wed Apr 27 20:44:26 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> Message-ID: <20050427184418.GE25148@caltech.edu> -> > Martijn Faassen: -> > One can hardly expect people to drop the frameworks they've invested a -> > lot in and all converge upon another one. You can't even expect Python -> > programmers to actually stop creating new frameworks. -> -> Expect? No. Ask? Yes. Believe that if they don't, they'll be -> wondering in five years why Python has become the next Tcl, instead of -> the next Perl? Absolutely. *shudder* Anyway, doomsday thoughts aside: we don't need new frameworks, or even new amalgamations of existing projects. I think we need to make the existing frameworks easier to use. Python Paste and WSGI in general will help with this by standardizing deployment, but we need more and better documentation aimed at beginning users. IMO, of course. (This is one of the factors motivating my Quixote 2 tutorial.) cheers, --titus From ianb at colorstudy.com Wed Apr 27 21:10:02 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed Apr 27 21:12:15 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> Message-ID: <426FE38A.9020300@colorstudy.com> Greg Wilson wrote: > > > Greg Wilson wrote: > >> > Well, here we all are, two weeks later... > > > Ian Bicking: > > [Subway on Paste] > > [Quixote on Paste] > > [Adrian Holovaty's stuff on Paste] > > There's an old Chinese proverb (well, I was told it's Chinese): "If a > hundred sages each tell you something different, the odds are that > they're all wrong." I think WSGIKit/Paste is a great idea, but as long > as a dozen people keep saying, "Use mine! No, use mine!", the average > "need to get it done for Tuesday" web developer isn't going to touch any > of them. In part it's because we're trying to answer the wrong question: "How should one do web programming in Python." How boring. We should be offering answers to things like "how do I create a web frontend to my existing database?" or "how do I provide a rich interactions to users?" or "how do I track incoming issues in my organization?" Or whatever. FWIW, I strongly suggested to Adrian that he call his project a CMS, not a web framework. Sure, it is also a web framework, but who cares about another one of those? It's interesting because it's a CMS, and it fills a niche that isn't well filled right now. I think that's how we should market our respective efforts generally, as systems to address specific problems that aren't related to Python. If the problems are related to Python, then we don't have to convince anyone to use Python to solve them, do we? Of course, I fall into this as well, as Paste is oriented towards the Python world. But then, I'm comfortable if things built on Paste are advertised without mentioning Paste, just as they are advertised without mentioning Distutils or the zip file format. My hope with Paste is that it helps chip away at the differences between these efforts. There's a lot of uninteresting differences between frameworks, and I hope that Paste will grow to fill those parts of the stack. I think that will help clarify important differences, where frameworks address different niches and deserve to coexist. > > Martijn Faassen: > > One can hardly expect people to drop the frameworks they've invested a > > lot in and all converge upon another one. You can't even expect Python > > programmers to actually stop creating new frameworks. > > Expect? No. Ask? Yes. Believe that if they don't, they'll be > wondering in five years why Python has become the next Tcl, instead of > the next Perl? Absolutely. Perl is already shaping up to be the next Tcl, IMHO. But anyway, let's not be *too* melodramatic here. If one framework pulls out ahead, it won't be because it sniped all the users of other Python frameworks. The numbers aren't large enough for sniping to gain you much. It'll be because it gets Perl, PHP, Java, and VB/.NET developers. Because it grabs people who didn't even know that deep down they were looking for Python. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Wed Apr 27 21:16:33 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed Apr 27 21:18:43 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <20050427184418.GE25148@caltech.edu> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> <20050427184418.GE25148@caltech.edu> Message-ID: <426FE511.7000700@colorstudy.com> Titus Brown wrote: > Anyway, doomsday thoughts aside: we don't need new frameworks, or even > new amalgamations of existing projects. I think we need to make the > existing frameworks easier to use. Python Paste and WSGI in general > will help with this by standardizing deployment, but we need more and > better documentation aimed at beginning users. One of the things I've been experimenting with in Paste is fully doctested tutorials. I'm optimistic about it -- certainly the structure needs to be better filled out, but I think it is useful. WSGI and some of the other Paste utilities help quite a bit. I always feel like when I write docs and tutorials I'm planting a bulb that will grow into a beautiful flower, but usually the garden gets paved over before spring. Heh, that's a stupid metaphor. But anyway, if documentation can be cumulative instead of temporary it makes a huge difference, and testable documentation is necessary for that. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From david at sundayta.com Wed Apr 27 23:27:22 2005 From: david at sundayta.com (Dave Warnock) Date: Wed Apr 27 23:27:22 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <20050427184418.GE25148@caltech.edu> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> <20050427184418.GE25148@caltech.edu> Message-ID: <427003BA.4000802@sundayta.com> Titus, > Anyway, doomsday thoughts aside: we don't need new frameworks, or even > new amalgamations of existing projects. I think we need to make the > existing frameworks easier to use. Python Paste and WSGI in general > will help with this by standardizing deployment, but we need more and > better documentation aimed at beginning users. > > IMO, of course. > > (This is one of the factors motivating my Quixote 2 tutorial.) All agreed and I liked the tutorial. Dave From david at sundayta.com Wed Apr 27 23:27:52 2005 From: david at sundayta.com (Dave Warnock) Date: Wed Apr 27 23:27:51 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <426FE511.7000700@colorstudy.com> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> <20050427184418.GE25148@caltech.edu> <426FE511.7000700@colorstudy.com> Message-ID: <427003D8.6090306@sundayta.com> Ian, > One of the things I've been experimenting with in Paste is fully > doctested tutorials. I'm optimistic about it I like what you have been doing with this. Looks a really good use of doctest > But anyway, if documentation can be > cumulative instead of temporary it makes a huge difference, and testable > documentation is necessary for that. Agreed. Dave From jjinux at gmail.com Thu Apr 28 07:33:08 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Thu Apr 28 07:33:11 2005 Subject: [Web-SIG] Just lost another one to Rails In-Reply-To: References: Message-ID: Dear Greg, I've read your email. What's your solution? Best Regards, -jj On 4/8/05, Greg Wilson wrote: > A colleague down in the States, whom I converted to Python two years > ago, sent me mail last night saying, "Well, we've decided to go with > Rails. Means learning a new language, but their stuff just plain worked > out of the box." He's a bright guy --- used to run an ISP in the > mid-90s, and those of you who were involved in Apache between '96 and > '01 would probably recognize his name. > > So my question is, has there been any movement in the wake of Michelle > Levesque's web-off, and all the press Rails has been getting, to do > something about this? -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From titus at caltech.edu Thu Apr 28 10:22:22 2005 From: titus at caltech.edu (Titus Brown) Date: Thu Apr 28 10:22:26 2005 Subject: [Web-SIG] WSGI adapter for Quixote 2.0. Message-ID: <20050428082222.GA3671@caltech.edu> Hi all, pursuant to my Quixote2 tutorial and a private request, I've updated my WSGI wrapper for Quixote to work for Quixote 2.0. It's been greatly simplified in the process. It's called QWIP2, and there's a not-very-updated Web page here: http://www.idyll.org/~t/www-tools/wsgi/ The source is directly available here: http://darcs.idyll.org/~t/projects/qwsgi/quixote2/qwip2.py and it's running the Quixote 2 demo inside of CGI here: http://issola.caltech.edu/~t/test-cgi/qx2-demo.cgi/ QWIP2 is even simpler to use than QWIP: --- import qwip2 app_obj = qwip2.QWIP2(create_publisher_fn) --- so e.g. for the demo I just do --- import qwip2 import quixote.demo.mini_demo app_obj = qwip2.QWIP2(quixote.demo.create_publisher) run_with_cgi(app_obj) --- It seems to work without any problems, but I don't have any automated tests for Quixote 2.0 yet, so I can't be sure. Still, Quixote is pretty regular in its output so I doubt there are any naster corner cases. cheers, --titus From faassen at infrae.com Thu Apr 28 12:00:54 2005 From: faassen at infrae.com (Martijn Faassen) Date: Fri Apr 29 02:32:50 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <426FE38A.9020300@colorstudy.com> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> <426FE38A.9020300@colorstudy.com> Message-ID: <4270B456.4080800@infrae.com> Ian Bicking wrote: [snip] > FWIW, I strongly suggested to Adrian that he call his project a CMS, not > a web framework. Sure, it is also a web framework, but who cares about > another one of those? It's interesting because it's a CMS, and it fills > a niche that isn't well filled right now. Could you identify this niche a bit more clearly? There is one huge Python-based CMS called Plone (which in my experience enjoys much the same type of buzz as Ruby on Rails does), and there are others such as Silva (which I've helped build) and CPS. To my eyes this niche is very well filled. Is the difference that all these efforts are Zope based? Is the niche a non-Zope-based Python-based CMS? CMSes are somewhat interesting as they're applications and frameworks at the same time. That is, they provide a very visible end-user user interface, but for every deployment they also need integration, customization and so on. Zope, for all its flaws, has a now fairly-well understood (by Zope developers) infrastructure to do this kind of stuff. Because Zope and Zope-based CMSes have an explicit user interface they also tend to attract users that are not Python developers but do want to do tasks like create new forms, new types of content objects, and the like, and sysadmins that do not know Python that need to keep the whole system running and well-configured. I haven't seen Andrew Kuchling explain what he meant by his implication that Zope (3) should not be using Python, but I think it indicates that there is significantly different perspective involved here. With all the focus on extensible CMSes, Zope developers are perhaps more used to applications that are also frameworks and that expose traditionally development-level functionality to non-developers than most other people who build Python-based web applications, and therefore the customizability requirements may be vastly different. It seems therefore to me that a dialog between Zope and the rest of the Python world might be very useful. The Zope 3 project has been trying to get off the island that Zope 2 is and into the wider Python world, but has only been partially successful so far. I think the Python web development world can learn a lot from the Zope perspective though, even though the opposite direction (Zope learning and reusing from the wider Python world) is even more important. Regards, Martijn From faassen at infrae.com Thu Apr 28 11:42:00 2005 From: faassen at infrae.com (Martijn Faassen) Date: Fri Apr 29 02:32:51 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> Message-ID: <4270AFE8.7050200@infrae.com> Greg Wilson wrote: > > > Greg Wilson wrote: > >> > Well, here we all are, two weeks later... > > > Ian Bicking: > > [Subway on Paste] > > [Quixote on Paste] > > [Adrian Holovaty's stuff on Paste] > > There's an old Chinese proverb (well, I was told it's Chinese): "If a > hundred sages each tell you something different, the odds are that > they're all wrong." I think WSGIKit/Paste is a great idea, but as long > as a dozen people keep saying, "Use mine! No, use mine!", the average > "need to get it done for Tuesday" web developer isn't going to touch any > of them. I think the best way to evolve away from this is to try to reuse code. I think what Paste is doing is an example of trying to do this. I'd rather see this as "two weeks later", you come back, and Zope 3 is using Twisted and WSGI, and Paste uses WSGI to integrate with Subway, Quixote, and so on. So who knows what the next two weeks will bring? :) > > Martijn Faassen: > > One can hardly expect people to drop the frameworks they've invested a > > lot in and all converge upon another one. You can't even expect Python > > programmers to actually stop creating new frameworks. > > Expect? No. Ask? Yes. Believe that if they don't, they'll be > wondering in five years why Python has become the next Tcl, instead of > the next Perl? Absolutely. Such drama and doom ahead... I'm not sure whether making Python "the next Perl" is something good to aim for anyway... Trends come and pass, and eventually we'll see people complain about Rails and rave about something else. I've seen quite similar blurbs with Zope 2 back in '00 or so, for instance. Meanwhile Zope has grown up a lot, especially in Europe, where dozens of companies make their money building on top of the Zope platform. This may yet happen with Ruby on Rails, but let's not ignore the real business happening with other systems. That's not to say we shouldn't do anything, but going "we must DO something NOW! No, what you did is NOT IT, we must DO SOMETHING!" is starting to feel less constructive at this stage. What are you *asking*? What are you suggesting is done? Regards, Martijn From ianb at colorstudy.com Thu Apr 28 20:02:31 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 29 02:33:07 2005 Subject: [Web-SIG] Preferred set of WSGI servers Message-ID: <42712537.2010805@colorstudy.com> Here's the full set of WSGI-enabled servers I'd like to see (and in some way encorporate into Paste, of course): * CGI gateway (done: not sure of canonical location) * Simple threaded HTTP server (done: wsgiutils) * mod_python (?) * ISAPI (done: http://isapi-wsgi.python-hosting.com/) * Zope product (not done) Those are all rather specific to certain environments. Then I'd like to see two other servers: multi-process and threaded. Obviously the two imply very different architectures. Well, maybe not, but it seems like it to me. Right now we have several servers with different purposes: SCGI (one for forking and threaded each), FastCGI, HTTP, etc. I don't really like that, I feel protocol support should be a separate concern. I would like for the two "preferred" deployment servers to support this set of protocols: * HTTP * HTTPS * FastCGI * SCGI Support for other protocols like FTP doesn't hurt, of course, but their exact relation to WSGI is unclear. Right now servers seem to be FastCGI-ish (which includes SCGI) *or* HTTP. I don't think this protocol choice should restrict the server choices. I think FastCGI-ish setups can be easier to configure an integrate; other people may disagree, but I don't see any technical reason not to include all these protocols, so why not make everyone happy? (Note that I'm personally not concerned about the fancier features of FastCGI specifically, like starting processes and doing pooling as part of the protocol; I'm only interested in the delegation of requests to different processes.) Twisted is an obvious contender for the threaded server. However, to my knowledge it does not support FastCGI or SCGI (as a client). Also, WSGI support is only available for twisted.web2, which doesn't have a release yet. Lastly, the installation isn't as easy as it could be -- it requires Zope interfaces, which includes C extensions. So, there's some work to be done there. To their credit, with the recent split of Twisted into smaller packages they are moving in the right direction. Medusa is a simpler alternative; it's not as actively maintained, but there aren't as many packaging or dependency issues. For the forking server, I really have no idea. There's several out there, but I don't know about all the features that are desirable or implemented. Several of the servers don't speak HTTP. I can't make any specific criticisms, because I don't even know what to look at. Lingering in the background of these server discussions are issues like user isolation and module reloading (and/or server restarting). Thoughts, reactions? From ianb at colorstudy.com Thu Apr 28 17:11:51 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 29 02:33:15 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <4270B456.4080800@infrae.com> References: <200504150811.AA65798380@boddie.org.uk> <42652961.70108@infrae.com> <200504191858.07820.paul@boddie.org.uk> <42654C1C.50708@infrae.com> <426FE38A.9020300@colorstudy.com> <4270B456.4080800@infrae.com> Message-ID: <4270FD37.4070602@colorstudy.com> Martijn Faassen wrote: > Ian Bicking wrote: > [snip] > >> FWIW, I strongly suggested to Adrian that he call his project a CMS, >> not a web framework. Sure, it is also a web framework, but who cares >> about another one of those? It's interesting because it's a CMS, and >> it fills a niche that isn't well filled right now. > > > Could you identify this niche a bit more clearly? There is one huge > Python-based CMS called Plone (which in my experience enjoys much the > same type of buzz as Ruby on Rails does), and there are others such as > Silva (which I've helped build) and CPS. To my eyes this niche is very > well filled. Is the difference that all these efforts are Zope based? Is > the niche a non-Zope-based Python-based CMS? Well, it's not my product to define -- it's not even released yet -- but I wouldn't define it as non-Zope. It's purely RBMS-backed, and all customization is done by a programmer. In focus maybe it's more like OpenACS (at least my vague idea of what OpenACS does). These are very different than the Zope CMSs, and I think it's valid niche. "CMS" is a vague concept, and there's room for a large number of differentiated products. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From gvwilson at cs.utoronto.ca Thu Apr 28 17:18:54 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Fri Apr 29 02:43:07 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: Message-ID: > Shannon -jj Behrens wrote: > Dear Greg, > I've read your email. What's your solution? > Best Regards, > -jj Hi JJ. My solution is for Guido (or someone with equivalent authority) to appoint someone "Benevolent Dictator for the Web for One Year", with a mandate to put together something that has all the features that are getting Rails so much attention. Whatever s/he comes up with in that year will then be shipped as part of the 2.X release of Python in 2006. I think Ian Bicking or Michelle Levesque would be good choices for the role: they know the field, and they're not emotionally committed to any of the existing frameworks. Thanks, Greg From mark.john.rees at gmail.com Fri Apr 29 03:38:38 2005 From: mark.john.rees at gmail.com (Mark Rees) Date: Fri Apr 29 03:38:40 2005 Subject: [Web-SIG] Preferred set of WSGI servers In-Reply-To: <42712537.2010805@colorstudy.com> References: <42712537.2010805@colorstudy.com> Message-ID: On 4/29/05, Ian Bicking wrote: > Here's the full set of WSGI-enabled servers I'd like to see (and in > some way encorporate into Paste, of course): > > * CGI gateway (done: not sure of canonical location) > * Simple threaded HTTP server (done: wsgiutils) > * mod_python (?) There is http://www.c-area.ch/code/mp_wsgi_handler.py but the licence may be an issue. It is based on this email from the list: http://mail.python.org/pipermail/web-sig/2004-October/000980.html > * ISAPI (done: http://isapi-wsgi.python-hosting.com/) Currently looking at best way of getting isapi-wsgi to work with paste. Regards Mark From smulloni at smullyan.org Fri Apr 29 06:37:16 2005 From: smulloni at smullyan.org (Jacob Smullyan) Date: Fri Apr 29 06:37:19 2005 Subject: [Web-SIG] Preferred set of WSGI servers In-Reply-To: <42712537.2010805@colorstudy.com> References: <42712537.2010805@colorstudy.com> Message-ID: <20050429043716.GA26647@smullyan.org> On Thu, Apr 28, 2005 at 01:02:31PM -0500, Ian Bicking wrote: > For the forking server, I really have no idea. There's several out > there, but I don't know about all the features that are desirable or > implemented. Several of the servers don't speak HTTP. I can't make > any specific criticisms, because I don't even know what to look at. I noticed that Alan Saddi has a adaptive preforking server implementation in flup; it looks good. SkunkWeb's preforking server, which I am beginning to play with in the to-be-wsgi skunk4 branch, isn't adaptive (by choice); it might become non-adaptive by default but capable of being configured to be adaptive. (For an app server where you may have relatively heavy processes and be hanging on to database connections in every connection, forking new processes as a function of load can make a bad problem worse.) My skunk4 sandbox in skunkweb subversion has the base server modules, but no wsgi integration yet; I'm distracted with other projects at the moment, but I'll be cycling back to skunk4 soon, and will make a point of writing a server compatible with paste. > Lingering in the background of these server discussions are issues > like user isolation and module reloading (and/or server restarting). I don't see how single-process wsgi servers could provide user isolation, if I understand you aright, except by proxying to other processes, but a sort of suexec could conceivably be implemented even as a middleware, if anyone feels like impaling themselves on that project and being blamed for every resulting problem. As for reload/restart, are there any mysterious issues here? Are you concerned about the implementation problem of getting reloading to work, possibly at times other than at restart as well as at restart, without broken references dangling, or the policy issue of how to try to standardize restart behaviors for the different servers with Paste (namely, how do they react to HUP, and if they restart, what do they do to sys.modules), or something else? Your clever reloader module seems to address the problem of reloading for development convenience; I suppose it might occasionally break things, but it's voluntary. I do think it would be desirable to have more or less the same signal handling for all the backends, although this may not be the most exciting thing to work on right now; a generic /etc/init.d/paste script shouldn't have to care which backend you are calling, it should just know that it has a pid file it can do the standard things to. Some refactoring ideas about the server layer occurred to me that might be applicable in future and make things a bit more robust: 1. The assumption (am I right it that there is one?) that servers only serve one port at a time should probably be loosened at some point. 2. Support for running on UNIX sockets and privileged TCP ports (hence, dropping privileges) would also be a boon. 3. It would be cleaner and provide more options if the server process model and the particular lingo the server might talk over a particular socket (http, scgi, fcgi, mod_skunk, etc.) were separated. My approach would be to nest wsgi apps into generic jobs of speaking a lingo, and associate each lingo-job with a list of socket addresses; that addresses the first point, too. Cheers, js -- Jacob Smullyan -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/web-sig/attachments/20050429/feb40964/attachment-0001.pgp From renesd at gmail.com Fri Apr 29 07:26:18 2005 From: renesd at gmail.com (Rene Dudfield) Date: Fri Apr 29 07:26:21 2005 Subject: [Web-SIG] Preferred set of WSGI servers In-Reply-To: <42712537.2010805@colorstudy.com> References: <42712537.2010805@colorstudy.com> Message-ID: <64ddb72c05042822267c0a0696@mail.gmail.com> On 4/29/05, Ian Bicking wrote: > Here's the full set of WSGI-enabled servers I'd like to see (and in > some way encorporate into Paste, of course): > > * CGI gateway (done: not sure of canonical location) > * Simple threaded HTTP server (done: wsgiutils) > * mod_python (?) > * ISAPI (done: http://isapi-wsgi.python-hosting.com/) > * Zope product (not done) > > Those are all rather specific to certain environments. Then I'd like > to see two other servers: multi-process and threaded. Obviously the > two imply very different architectures. Well, maybe not, but it seems > like it to me. > > Right now we have several servers with different purposes: SCGI (one > for forking and threaded each), FastCGI, HTTP, etc. I don't really > like that, I feel protocol support should be a separate concern. I > would like for the two "preferred" deployment servers to support this > set of protocols: > > * HTTP > * HTTPS > * FastCGI > * SCGI > > Support for other protocols like FTP doesn't hurt, of course, but > their exact relation to WSGI is unclear. Right now servers seem to be > FastCGI-ish (which includes SCGI) *or* HTTP. I don't think this > protocol choice should restrict the server choices. I think > FastCGI-ish setups can be easier to configure an integrate; other > people may disagree, but I don't see any technical reason not to > include all these protocols, so why not make everyone happy? (Note > that I'm personally not concerned about the fancier features of > FastCGI specifically, like starting processes and doing pooling as > part of the protocol; I'm only interested in the delegation of > requests to different processes.) > > Twisted is an obvious contender for the threaded server. However, to > my knowledge it does not support FastCGI or SCGI (as a client). Also, > WSGI support is only available for twisted.web2, which doesn't have a > release yet. Lastly, the installation isn't as easy as it could be -- > it requires Zope interfaces, which includes C extensions. So, there's > some work to be done there. To their credit, with the recent split of > Twisted into smaller packages they are moving in the right direction. > Medusa is a simpler alternative; it's not as actively maintained, but > there aren't as many packaging or dependency issues. > Twisted reverse proxied by apache2 is my server of choice at the moment for wsgi. As it can nicely run multiple wsgi apps. I use apache2 instead of lighttpd now, because it supports compression, and caching of dynamic content. With twisted I can also do other things in the background quite easily. Like making http requests, sending/serving email, etc etc. There are still a few bugs in twisted.web2 as it is not released. However there are a few active contributors to web2 who are quick to help out with any problems. Here is an example of using twisted to serve two wsgi apps with web2. http://www.madecollective.com/~rene/two_wsgi.py > For the forking server, I really have no idea. There's several out > there, but I don't know about all the features that are desirable or > implemented. Several of the servers don't speak HTTP. I can't make > any specific criticisms, because I don't even know what to look at. > http://www.saddi.com/software/py-lib/ with his flup package contains experimental forking/preforking wsgi servers. Have you seen these? The above also contains an XMLRPC wsgi server. So you can easily serve xmlrpc on with wsgi too :) > Lingering in the background of these server discussions are issues > like user isolation and module reloading (and/or server restarting). > > Thoughts, reactions? > Another nice thing about twisted is that it has server restarting in there. It also has module reloading. Allthough I don't use either. I think a separate process which monitors things and sends the app restart, reload, signals is a good idea. Then in development you can have the monitor send it a restart signal as soon as code/files change. I use this bash script on debian linux to watch a directory and then do stuff if any of my files change. It uses rsync, which is really fast. It doubles as an automatic incremental backup service. With putting files that have changed in separate time stamped directories. It can be easily modified to save the backups on another machine too. watcher.sh #!/bin/bash DIR_TO_WATCH=pretendpaper/pp BACKUP_DIR=/tmp/ CHANGED_DIR=/tmp/changed/ export THE_DATE=`date +%Y_%m_%d_%H_%M_%s` # you need to make sure the changed dir is writable, with good permissions. # if you use /tmp uncomment these lines. #mkdir -p $CHANGED_DIR #chmod 755 $CHANGED_DIR NUM_CHANGED=`rsync --delete --backup --backup-dir=$CHANGED_DIR/$THE_DATE --progress -v -a $DIR_TO_WATCH $BACKUP_DIR | grep "100%" | wc -l` if [ "$NUM_CHANGED" = "0" ]; then echo "nothing changed" else echo "something changed" # Put your commands here. eg. rebuild cheetah templates, restart server, flush caches etc. fi Then I use `watch -n 2 ./watcher.sh` in an xterm. Which makes it run every two seconds. User isolation: ---------------------- Running different processes for different users is the best way I think. That way you can use operating system level resource constraints and auditing. On pretendpaper.com I have given users with different roles separate processes. Each process also uses a different database user, to add in another layer of security. So basic members database user does not have access to select/delete/update tables, the process serving their requests should not be able to access them either. On linux(and other unixes) with ulimit you can set each group of users having a different amount of available memory and cpu usage. eg ulimit -v 50000 will limit it to running 50 megabytes of virtual memory. Cheers, Rene Dudfield e: "@".join( reversed(['madecollective.com', 'rene']) ) w: http://www.madecollective.com/ w: http://www.pretendpaper.com/ For creative, and technical services contact us @ made. From floydophone at gmail.com Fri Apr 29 15:18:11 2005 From: floydophone at gmail.com (Peter Hunt) Date: Fri Apr 29 15:18:14 2005 Subject: [Web-SIG] Re: Preferred set of WSGI servers Message-ID: <6654eac405042906187d99b6fd@mail.gmail.com> Actually, I think a mod_wsgi for Apache and IIS would be one of _the_ most important things for WSGI. I think that it would search for a __wsgi__.py file (or maybe something with a better name) which would expose a WSGI application named "application" that would handle requests for the directory in which it exists. This would allow a user to simply drop a file on the webserver and have it work. It would be excellent for shared hosting solutions. Thoughts? Peter Hunt -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/web-sig/attachments/20050429/8f74d8e4/attachment.htm From mike_mp at zzzcomputing.com Fri Apr 29 16:24:04 2005 From: mike_mp at zzzcomputing.com (mike bayer) Date: Fri Apr 29 16:25:18 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: Message-ID: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> > I think Ian Bicking or Michelle Levesque would be good choices for the > role: they know the field, and they're not emotionally committed to any > of the existing frameworks. Michelle is currently a computer science undergrad at the university of Toronto. Are you saying all the people with many years of real-world experience building dynamic web applications and sites for the corporate world and the educational world should be forced to obey the technical direction of a college student with no professional experience ? From gvwilson at cs.utoronto.ca Fri Apr 29 16:51:32 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Fri Apr 29 17:06:45 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> Message-ID: >>Greg Wilson wrote: >>I think Ian Bicking or Michelle Levesque would be good choices for the >>role... > mike bayer wrote: > Michelle is currently a computer science undergrad at the university of > Toronto. Are you saying all the people with many years of real-world > experience building dynamic web applications and sites for the corporate > world and the educational world should be forced to obey the technical > direction of a college student with no professional experience ? Greg Wilson wrote: Nope, I didn't say that at all (and I don't think that putting words in people's mouths helps this discussion at all). This is open source: no one can "force" anyone to do anything, not even Guido. I mentioned Ian and Michelle because they know the players, have worked with most of the candidate systems, and have demonstrated that they're willing to listen as well as talk; there are undoubtedly other people who could "play Guido" just as well. Thanks, Greg p.s. Ka-Ping Yee was "just an undergrad" when he started contributing to Python; I don't remember anyone thinking that ought to disqualify him. From mike_mp at zzzcomputing.com Fri Apr 29 17:33:54 2005 From: mike_mp at zzzcomputing.com (mike bayer) Date: Fri Apr 29 17:35:34 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> Message-ID: <32107.66.192.34.8.1114788834.squirrel@66.192.34.8> I am also opposed to the role as a whole. Guido created Python, established himself as the obvious controller of its direction, and then because he was really good at it, the developers came and it became popular. All of us, by choosing to be Python developers, have indivdually chosen to be part of a community directed by his guidance. But if someone new is then artifically assigned, without the benefit of the individual choice we have all made, a similar ability to approve or reject architectural ideas among that same set of developers, who now have lots of time and effort invested in the community, towards the goal of setting the direction for "how will (mostly) everyone do Python web development next year" as well as "what approaches will most of our customers and employers expect us to use next year", the decisions of that direction most certainly will be imposed upon a lot of us; with our own individual ideas and frameworks marginalized, the only option left is to find another community. As we are all hoping the Ruby programming language is not imposed upon us in the coming years via a similar mechanism, we would also hope that our own community doesnt decide to internally squash a large portion of its ideas and diversity, championing just a small subset of them, through no less than a dictatorship-driven process. Also, lets think about what the role really means: we must choose someone to oversee the development of a ripoff of some other language's web platform and establish it as the "one true way to develop web applications in Python", based solely on the reason that the original has been popular among junior and mid-level web developers for about 6 months, despite the fact that its hardly in large-scale production usage anywhere. Its like choosing to do your company's website using Ruby on Rails, with all the risks associated with using a new platform, mulitplied by 10,000 since its the approach now foisted upon the entire language community as a whole. I just find it hard to be optimistic about development that is motivated by pure reactivity. > >>Greg Wilson wrote: >>>I think Ian Bicking or Michelle Levesque would be good choices for the >>>role... > >> mike bayer wrote: >> Michelle is currently a computer science undergrad at the university of >> Toronto. Are you saying all the people with many years of real-world >> experience building dynamic web applications and sites for the corporate >> world and the educational world should be forced to obey the technical >> direction of a college student with no professional experience ? > > Greg Wilson wrote: > Nope, I didn't say that at all (and I don't think that putting words in > people's mouths helps this discussion at all). This is open source: no > one can "force" anyone to do anything, not even Guido. I mentioned Ian > and Michelle because they know the players, have worked with most of the > candidate systems, and have demonstrated that they're willing to listen > as well as talk; there are undoubtedly other people who could "play > Guido" just as well. > > Thanks, > Greg > > p.s. Ka-Ping Yee was "just an undergrad" when he started contributing to > Python; I don't remember anyone thinking that ought to disqualify him. > > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/mike_mp%40zzzcomputing.com > From gvwilson at cs.utoronto.ca Fri Apr 29 17:39:02 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Fri Apr 29 17:40:31 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <32107.66.192.34.8.1114788834.squirrel@66.192.34.8> References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> <32107.66.192.34.8.1114788834.squirrel@66.192.34.8> Message-ID: > mike bayer wrote: > [...] > Also, lets think about what the role really means: we must choose someone > to oversee the development of a ripoff of some other language's web > platform and establish it as the "one true way to develop web applications > in Python" Greg Wilson writes: Mike, as I said in my previous message, I don't think that putting words in my mouth helps this discussion at all. Thanks, Greg From mike_mp at zzzcomputing.com Fri Apr 29 17:51:04 2005 From: mike_mp at zzzcomputing.com (mike bayer) Date: Fri Apr 29 17:52:17 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> <32107.66.192.34.8.1114788834.squirrel@66.192.34.8> Message-ID: <40797.66.192.34.8.1114789864.squirrel@66.192.34.8> ok, well its not fair to say im putting words in your mouth. I am obviously re-stating them, in the way they are striking me as what result they would ultimately indicate, which is the part that is my opinion: your words: "with a mandate to put together something that has all the features that are getting Rails so much attention. " my words: "oversee the development of a ripoff of some other language's web platform" (are all those features getting Rails so much attention really that worthy in the long term?) your words: "Whatever s/he comes up with in that year will then be shipped as part of the 2.X release of Python in 2006." my words: "establish it as the "one true way to develop web applications in Python"" (as there is no web framework shipped with Python right now, it would be pretty dramatic if a certain approach all the sudden was) From todd at slack.net Fri Apr 29 18:02:01 2005 From: todd at slack.net (Todd Grimason) Date: Fri Apr 29 17:56:36 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <40797.66.192.34.8.1114789864.squirrel@66.192.34.8> References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> <32107.66.192.34.8.1114788834.squirrel@66.192.34.8> <40797.66.192.34.8.1114789864.squirrel@66.192.34.8> Message-ID: <20050429160201.GA27533@detroit.slack.net> * mike bayer [2005-04-29 11:57]: Just to emerge from lurking for a moment, and without years of working in the python community like you guys, it seems pretty obvious to me that such a solution (a "dictated" web app approach) would never happen. >From what I've read of Guido's this seems completely at odds with his views/approach, not to mention the wide revolt it would cause. (back to lurking) -- ______________________________ toddgrimason*todd-AT-slack.net From jhylton at gmail.com Fri Apr 29 17:58:53 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Fri Apr 29 17:58:55 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: Message-ID: On 4/28/05, Greg Wilson wrote: > My solution is for Guido (or someone with equivalent authority) > to appoint someone "Benevolent Dictator for the Web for One Year", with > a mandate to put together something that has all the features that are > getting Rails so much attention. Whatever s/he comes up with in that > year will then be shipped as part of the 2.X release of Python in 2006. I don't think this is a good idea for several reasons. Let's imagine we could go back in time four years and tell the Ruby community the same thing: Appoint someone to research a popular new way of building web applications and add that to the next release of Ruby. What would the result have been for Ruby? Would it now look like PHP? I don't know, but I'm skeptical of a solution that copies an existing popular tool. I don't think a large web programming toolkit belongs in the Python distribution. If anything, go the other way around and package a particular version of Python with this web toolkit. I'm also skeptical of a plan that sets out to build the one right way that everyone will use. I don't know anything about the history of Rails, but I'm guessing that the project didn't start because Matz said "Build the official web toolkit for Ruby users" and it didn't become successful because Ruby programmers adopted it. That is, I imagine it became popular because people liked it and not because people agreed in advance that the would like it. Jeremy PS None of the top 10 Google results for "web programming" mentioned Python (PHP is second). Four of the top 10 results for "web programming frameworks" mention Python, including Ian's blog post about why web programming matters. Maybe we'd do better if we didn't have the frameworks <0.2 wink>. From jhylton at gmail.com Fri Apr 29 18:07:06 2005 From: jhylton at gmail.com (Jeremy Hylton) Date: Fri Apr 29 18:07:09 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: Message-ID: Oops. Slight clarification: On 4/29/05, Jeremy Hylton wrote: > I'm also skeptical of a plan that sets out to build the one right way > that everyone will use. I don't know anything about the history of > Rails, but I'm guessing that the project didn't start because Matz > said "Build the official web toolkit for Ruby users" and it didn't > become successful because Ruby programmers adopted it. That is, I > imagine it became popular because people liked it and not because > people agreed in advance that the would like it. ... and not because people agreed in advance that they would like it. Also, I imagine that Rails became popular enough that it lead people to Ruby rather than the other way around. Of course, Zope has been drawing people to Python for a long time; it just hasn't had much appeal as a general tool for Python programmers. Jeremy From todd at slack.net Fri Apr 29 18:13:00 2005 From: todd at slack.net (Todd Grimason) Date: Fri Apr 29 18:07:37 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <20050429160201.GA27533@detroit.slack.net> References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> <32107.66.192.34.8.1114788834.squirrel@66.192.34.8> <40797.66.192.34.8.1114789864.squirrel@66.192.34.8> <20050429160201.GA27533@detroit.slack.net> Message-ID: <20050429161300.GB27533@detroit.slack.net> Uh, just to clarify this was me, not mike, I forgot to kill that attribution line! * Todd Grimason [2005-04-29 12:02]: > * mike bayer [2005-04-29 11:57]: > > Just to emerge from lurking for a moment, and without years of working > in the python community like you guys, it seems pretty obvious to me > that such a solution (a "dictated" web app approach) would never happen. > >From what I've read of Guido's this seems completely at odds with his > views/approach, not to mention the wide revolt it would cause. > > (back to lurking) > -- ______________________________ toddgrimason*todd-AT-slack.net From gvwilson at cs.utoronto.ca Fri Apr 29 18:10:55 2005 From: gvwilson at cs.utoronto.ca (Greg Wilson) Date: Fri Apr 29 18:12:59 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: Message-ID: Hi Jeremy; thanks for your post. > Jeremy Hylton wrote: > > I don't think this is a good idea for several reasons. Let's imagine > we could go back in time four years and tell the Ruby community the > same thing: Appoint someone to research a popular new way of building > web applications and add that to the next release of Ruby. I'm not proposing that the BDFWFOY do research, or develop a new framework; I think we'd all agree we have enough already ;-). I *do* think that having someone facilitate (and referee) discussion aimed at pulling together a "no third step" combination of existing tools would be more useful than what we're doing right now. > I don't think a large web programming toolkit belongs in the Python > distribution. If anything, go the other way around and package a > particular version of Python with this web toolkit. Sure, both models have been successful in the past: PIL and Numeric are examples of "external, but only one" that have worked well. Perhaps someone who took part in the discussion about what XML tools to include in the core Python distro could chime in with a little history? > I'm also skeptical of a plan that sets out to build the one right way > that everyone will use. Well, then it's a good thing that's not what I'm asking for, isn't it? ;-) I think there's a place for entry-level (i.e., smaller than Zope) web app frameworks; the current confusion is about as helpful as having eight competing regular expression libraries, or six different "standards" for connecting to databases. Appointing a BDFWFOY is one way to solve the problem (though admittedly less popular than denying it exists ;-). I'd welcome others... Thanks, Greg p.s. BDFWFOY = Benevolent Dictator For the Web For One Year From faassen at infrae.com Fri Apr 29 18:54:47 2005 From: faassen at infrae.com (Martijn Faassen) Date: Fri Apr 29 18:49:52 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: Message-ID: <427266D7.3070302@infrae.com> Greg Wilson wrote: > > Jeremy Hylton wrote: >> I don't think a large web programming toolkit belongs in the Python >> distribution. If anything, go the other way around and package a >> particular version of Python with this web toolkit. > > > Sure, both models have been successful in the past: PIL and Numeric are > examples of "external, but only one" that have worked well. > > Perhaps someone who took part in the discussion about what XML tools to > include in the core Python distro could chime in with a little history? The stuff started as an external package first, and still is an external package (PyXML). In my opinion the whole XML in the core story sucks. Not because the code in the core is bad, but because there's another package, PyXML, that completely overrides it in a non-bug-compatible way, meaning you now need to test your code that uses the XML core-code both with and without (various versions of..) PyXML installed. If this is to be done for any hypothetical web framework, let's please not do it the XML way... This is not due to a mistake; at the time the core was deliberately made to look for PyXML. There are a ton of non-core XML frameworks around for Python, enjoying considerate popularity. The python 'xml' package is not the "one true way" to do XML with Python, and certainly doesn't enjoy anything near the popularity and buzz of Ruby on Rails, say. I don't see why the situation for any higher-level web framework should be different in this respect. >> I'm also skeptical of a plan that sets out to build the one right way >> that everyone will use. > > Well, then it's a good thing that's not what I'm asking for, isn't it? > ;-) I think there's a place for entry-level (i.e., smaller than Zope) > web app frameworks; the current confusion is about as helpful as having > eight competing regular expression libraries, or six different > "standards" for connecting to databases. Appointing a BDFWFOY is one > way to solve the problem (though admittedly less popular than denying it > exists ;-). I'd welcome others... What is exactly problem? That Ruby on Rails is taking mindshare away from Python? And so's PHP, and ASP, and Java, and so is Python from Java, and Zope from PHP, and so on. Could you explain why you consider Ruby on Rails to be different? Evidently you are of this opinion. I do see a problem with Python web frameworks, but it's got little to do with Ruby on Rails, though I do think we can learn from its marketing success. My problem is that of islands; people are often sitting in the island of their own framework and making it hard for people on their islands to work together. I think there's far more code we can share between frameworks than we've been doing in the past. More sharing code and integrate various bits using standard ways is good as it allows us to join forces and not waste time reinventing wheels. I think encouraging sharing, and setting up the circumstances to make sharing more easy (such as WSGI) is the only realistic way to reduce the confusion between Python web frameworks and make them start to evolve towards each other, which should also help marketing and reduce developer confusion during uptake. In the Zope world, we've got islands too, in the form of CMSes built on top of Zope, such as Plone, CPS and Silva. We're currently in the process of trying to get off our islands and converge, helped by the transition to Zope 3 component technologies. I've been helping with that process, and it's fairly hopeful. So now I've started to set my sights a bit further and I'm trying to help Zope itself become less of an island within the Python world. Regards, Martijn From ianb at colorstudy.com Fri Apr 29 18:53:30 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 29 18:56:14 2005 Subject: [Web-SIG] Re: Preferred set of WSGI servers In-Reply-To: <6654eac405042906187d99b6fd@mail.gmail.com> References: <6654eac405042906187d99b6fd@mail.gmail.com> Message-ID: <4272668A.1060706@colorstudy.com> Peter Hunt wrote: > Actually, I think a mod_wsgi for Apache and IIS would be one of _the_ > most important things for WSGI. I think that it would search for a > __wsgi__.py file (or maybe something with a better name) which would > expose a WSGI application named "application" that would handle requests > for the directory in which it exists. This would allow a user to simply > drop a file on the webserver and have it work. It would be excellent for > shared hosting solutions. Would it? From what I can tell, mod_php is very popular in shared hosts, but neither mod_python nor mod_perl are. I don't think mod_* makes it necessarily host-friendly. But from what I can tell of mod_python, it would be relatively easy to have something like .wsgi files, and give them a handler that runs them as WSGI apps. I can imagine a good setup for hosts being one based on forking per-user processes, which is adaptive primarily to scale down -- e.g., a largely dorman app could have 1 or even 0 processes running (at 0 it becomes similar to CGI, but presumably the process would stay around for some time to respond to subsequent requests). The "scaling down" scenario I often think about would be a email contact form -- one of those things that has to be an app, can be implemented and deployed separately from other aspects of the site, and yet it's clear waste of resources to keep a process always around to respond to such requests. Though it's actually someplace where CGI would work just fine; but lets say you don't want to educate the developer about when they might want to use other deployment strategies (which is a rather complex discussion really, that would be better avoided by providing one really good strategy and telling everyone to use it). Maybe mod_python is appropriate for that kind of situation, I'm not sure. It could also be implemented with a second, single process that was developed and deployed separately from Apache. As for IIS and ISAPI, I can't say I really know what the archicture of that system is, so I don't know. Oh, but that does remind me of another WSGI server: ASP. Though I remember vaguely this came up before and it seemed infeasible because ASP didn't support anything like PATH_INFO. Oh well, I can't remember now. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Fri Apr 29 19:15:31 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 29 19:18:13 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <427266D7.3070302@infrae.com> References: <427266D7.3070302@infrae.com> Message-ID: <42726BB3.7000500@colorstudy.com> Martijn Faassen wrote: > There are a ton of non-core XML frameworks around for Python, enjoying > considerate popularity. The python 'xml' package is not the "one true > way" to do XML with Python, and certainly doesn't enjoy anything near > the popularity and buzz of Ruby on Rails, say. I don't see why the > situation for any higher-level web framework should be different in this > respect. I think the standard library is difficult, because it mixes all sorts of things together. I involves methodology -- very conservative, backward compatible, slow to improve -- which suggests that a package should be at a certain point in its life. It also suggests applicability, that the package should be reasonably generic and not unduly complex. And it encourages people to use it, either directly or indirectly, instead of alternatives. All of these are important, but they are fairly separate. mxDateTime would have been a very capable implementation of date-time objects, and was a de facto standard, but because of methodology conflicts that didn't happen (I assume because mxDateTime had to be released on a schedule determined by commercial concerns). Wx is similar but even worse; there's no practical way I can imagine it going into the standard library, and it's not just inertia, but rather Tk actually *remains* ideal simply because it is largely dead. Generally the standard library is become much less practical way to move Python forward. Backporting has to be extensive for the library to be viable to use. Up-front design becomes burdensome -- I'm sure there was a lot of useful things that came out of the decimal discussion, but the whole process seemed odd and drawn-out from the outside. Copied designs can shortcut this (e.g., logging), but lead to other design problems. And even as this happens, distributions are often pulling the standard library apart into pieces. Batteries Included has certainly helped me as a developer, but it's not what I have my eyes on for the future. It would be nice if process and politics could be separated. There's a lot of modules in the standard library that no one should use, even if they aren't marked "deprecated" -- there might be no reason for those modules to go away ever, but they just shouldn't get new users. Similarly there's modules not in the standard library that should be de facto standards, but for good reason can't be part of the standard library development process. But there's no way to suggest that to people except for information question-and-answer sessions in IRC or mailing lists, or the the vague and unpredictable rankings of Google. "Politics" might be a bad name for this; it doesn't have to be contentious -- for instance, sgmllib has no hero who will be offended that people are steered away from it. But I can't think of a better name. Anyway, I think it might be possible to resolve that issue more easily than the technical issues involved in extending the standard library. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From steve at holdenweb.com Fri Apr 29 19:25:46 2005 From: steve at holdenweb.com (Steve Holden) Date: Fri Apr 29 19:25:46 2005 Subject: [Web-SIG] Re: Preferred set of WSGI servers In-Reply-To: <4272668A.1060706@colorstudy.com> References: <6654eac405042906187d99b6fd@mail.gmail.com> <4272668A.1060706@colorstudy.com> Message-ID: <42726E1A.6010809@holdenweb.com> Ian Bicking wrote: > Peter Hunt wrote: > >> Actually, I think a mod_wsgi for Apache and IIS would be one of _the_ >> most important things for WSGI. I think that it would search for a >> __wsgi__.py file (or maybe something with a better name) which would >> expose a WSGI application named "application" that would handle >> requests for the directory in which it exists. This would allow a user >> to simply drop a file on the webserver and have it work. It would be >> excellent for shared hosting solutions. > > > Would it? From what I can tell, mod_php is very popular in shared > hosts, but neither mod_python nor mod_perl are. I don't think mod_* > makes it necessarily host-friendly. But from what I can tell of > mod_python, it would be relatively easy to have something like .wsgi > files, and give them a handler that runs them as WSGI apps. > > I can imagine a good setup for hosts being one based on forking per-user > processes, which is adaptive primarily to scale down -- e.g., a largely > dorman app could have 1 or even 0 processes running (at 0 it becomes > similar to CGI, but presumably the process would stay around for some > time to respond to subsequent requests). The "scaling down" scenario I > often think about would be a email contact form -- one of those things > that has to be an app, can be implemented and deployed separately from > other aspects of the site, and yet it's clear waste of resources to keep > a process always around to respond to such requests. Though it's > actually someplace where CGI would work just fine; but lets say you > don't want to educate the developer about when they might want to use > other deployment strategies (which is a rather complex discussion > really, that would be better avoided by providing one really good > strategy and telling everyone to use it). > > Maybe mod_python is appropriate for that kind of situation, I'm not > sure. It could also be implemented with a second, single process that > was developed and deployed separately from Apache. > > As for IIS and ISAPI, I can't say I really know what the archicture of > that system is, so I don't know. Oh, but that does remind me of another > WSGI server: ASP. Though I remember vaguely this came up before and it > seemed infeasible because ASP didn't support anything like PATH_INFO. Oh > well, I can't remember now. > Seems to me that mod_python isn't really suitable for a shared-hosting environment because of the need to restart the server when indirectly-imported modules are updated, a situation that I believe mod_python does not detect. This would cause all other hosted sites' sessions to be forcibly terminated without some persistent-state mechanism that can't be guaranteed to be present for all sites, and of course most hosting companies aren't willing to have their customers say when shared-hosting servers are restarted. For these reasons I've never pestered any of our service providers to support mod_python even though I have done quite a bit of experimental work with it and really quite like that environment. regards Steve -- Steve Holden +1 703 861 4237 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/ From smulloni at smullyan.org Fri Apr 29 19:37:38 2005 From: smulloni at smullyan.org (Jacob Smullyan) Date: Fri Apr 29 19:37:41 2005 Subject: [Web-SIG] Re: Preferred set of WSGI servers In-Reply-To: <4272668A.1060706@colorstudy.com> References: <6654eac405042906187d99b6fd@mail.gmail.com> <4272668A.1060706@colorstudy.com> Message-ID: <20050429173738.GA16671@smullyan.org> On Fri, Apr 29, 2005 at 11:53:30AM -0500, Ian Bicking wrote: > I can imagine a good setup for hosts being one based on forking per-user > processes, which is adaptive primarily to scale down -- e.g., a largely > dorman app could have 1 or even 0 processes running (at 0 it becomes > similar to CGI, but presumably the process would stay around for some > time to respond to subsequent requests). The "scaling down" scenario I > often think about would be a email contact form -- one of those things > that has to be an app, can be implemented and deployed separately from > other aspects of the site, and yet it's clear waste of resources to keep > a process always around to respond to such requests. Though it's > actually someplace where CGI would work just fine; but lets say you > don't want to educate the developer about when they might want to use > other deployment strategies (which is a rather complex discussion > really, that would be better avoided by providing one really good > strategy and telling everyone to use it). Why prefer forking to threads in this case? -- Jacob Smullyan -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/web-sig/attachments/20050429/551e00b0/attachment-0001.pgp From mso at oz.net Fri Apr 29 19:47:16 2005 From: mso at oz.net (mso@oz.net) Date: Fri Apr 29 19:47:19 2005 Subject: [Web-SIG] Re: Preferred set of WSGI servers In-Reply-To: <4272668A.1060706@colorstudy.com> References: <6654eac405042906187d99b6fd@mail.gmail.com> <4272668A.1060706@colorstudy.com> Message-ID: <32805.161.55.66.227.1114796836.squirrel@www.oz.net> > Peter Hunt wrote: >> Actually, I think a mod_wsgi for Apache and IIS would be one of _the_ >> most important things for WSGI. I think that it would search for a >> __wsgi__.py file (or maybe something with a better name) which would >> expose a WSGI application named "application" that would handle requests >> for the directory in which it exists. This would allow a user to simply >> drop a file on the webserver and have it work. It would be excellent for >> shared hosting solutions. Ian Bicking wrote: > Would it? From what I can tell, mod_php is very popular in shared > hosts, but neither mod_python nor mod_perl are. I don't think mod_* > makes it necessarily host-friendly. But from what I can tell of > mod_python, it would be relatively easy to have something like .wsgi > files, and give them a handler that runs them as WSGI apps. More to the point, is mod_wsgi possible? I thought it would be a great replacement for mod_scgi and the framework-specific ones (mod_webkit), but banged my head against the wall when it came to 'wsgi.input' because you can't pickle an open file object or transmit it to another process. So mod_wsgi couldn't talk to an application server unless it were something like SCGI in disguise. I suppose it could work like mod_python with several parallel servers running in Apache child processes. But would be different enough from mod_python to justify a new name? More to the point, would it be reliable? I hear lots of problem reports about FastCGI and mod_python's reliability, but I don't hear such reports for mod_scgi, mod_webkit, or the various CGI adapters. For myself, mod_webkit has been rock solid except for one computer it wouldn't compile on, and likewise mod_scgi has been reliable except for a bug with certain Reload requests (when the user presses Reload), and I've never had a problem with a CGI adapter, but I would hesitate to use the others or something based on them until they get more reliability. mod_php is popular because it was marketed early on as the *only* way to run PHP, and is precompiled in many OS distributions. I doubt it's available on *most* public webhosts. Most webhosts offer only ASP or CGI if they offer any dynamic services at all. Getting them to support *any* Python mod_* option is an uphill battle in itself, so the existence of mod_wsgi won't really make a difference. Which doesn't mean it's not worth building for our own webservers. A reference implementation of mod_python + WSGI as described above would be nice, so we can see how feasable/reliable/efficient it is compared to the other adapters. Maybe it could replace all of them in market share. Or maybe we just need PHPython. :) -- Mike Orr From mike_mp at zzzcomputing.com Fri Apr 29 19:54:53 2005 From: mike_mp at zzzcomputing.com (mike bayer) Date: Fri Apr 29 19:56:08 2005 Subject: [Web-SIG] Re: Preferred set of WSGI servers In-Reply-To: <42726E1A.6010809@holdenweb.com> References: <6654eac405042906187d99b6fd@mail.gmail.com> <4272668A.1060706@colorstudy.com> <42726E1A.6010809@holdenweb.com> Message-ID: <44860.66.192.34.8.1114797293.squirrel@66.192.34.8> while this is not an exciting option, if separate per-user processes are needed, there is always apache going via mod_proxy to sub-instances of apache that run for each user's account on different ports, each running mod_python. im not sure how feasable that is to run many servers in various shared environments but I would think most other approaches involve some per-user daemon as well...and i think apache is pretty lightweight if configured so. >> Peter Hunt wrote: >> > Seems to me that mod_python isn't really suitable for a shared-hosting > environment because of the need to restart the server when > indirectly-imported modules are updated, a situation that I believe > mod_python does not detect. From ianb at colorstudy.com Fri Apr 29 20:08:36 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 29 20:10:54 2005 Subject: [Web-SIG] Re: Preferred set of WSGI servers In-Reply-To: <20050429173738.GA16671@smullyan.org> References: <6654eac405042906187d99b6fd@mail.gmail.com> <4272668A.1060706@colorstudy.com> <20050429173738.GA16671@smullyan.org> Message-ID: <42727824.1020502@colorstudy.com> Jacob Smullyan wrote: > On Fri, Apr 29, 2005 at 11:53:30AM -0500, Ian Bicking wrote: > >>I can imagine a good setup for hosts being one based on forking per-user >>processes, which is adaptive primarily to scale down -- e.g., a largely >>dorman app could have 1 or even 0 processes running (at 0 it becomes >>similar to CGI, but presumably the process would stay around for some >>time to respond to subsequent requests). The "scaling down" scenario I >>often think about would be a email contact form -- one of those things >>that has to be an app, can be implemented and deployed separately from >>other aspects of the site, and yet it's clear waste of resources to keep >>a process always around to respond to such requests. Though it's >>actually someplace where CGI would work just fine; but lets say you >>don't want to educate the developer about when they might want to use >>other deployment strategies (which is a rather complex discussion >>really, that would be better avoided by providing one really good >>strategy and telling everyone to use it). > > > Why prefer forking to threads in this case? Threading doesn't provide much isolation, so it's not really feasible for a multiuser situation. Well, Zope 2 managed it, but only with signficant heroics that also degrade the programming experience. OTOH, I don't know if it's useful to optimize for situations where most of the website is static with small dynamic portions. In a practical sense dynamicism (of some sort) is almost the norm these days (at least if a programmer is involved). So one threaded process per client may be reasonable as well. One of mod_webkit's underutilized features was the ability to drop .psp files anywhere, and have them processed in a single process. This is also possible with other configuration techniques (rewrite rules, LocationMatch, etc), but not popularized in other frameworks either. Probably in part because of the module/package orientation of Python, and it's less comfortable to be loading files from arbitrary locations. Explicit relative imports would be a real boon in this situation. Oh, and that brings up another point -- separate processes mean separate sys.paths and other isolation. Managing sys.path is a total pain in the butt. And module reloads of course, but even ignoring that sys.path is still annoying. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From foom at fuhm.net Fri Apr 29 20:18:28 2005 From: foom at fuhm.net (James Y Knight) Date: Fri Apr 29 20:18:35 2005 Subject: [Web-SIG] Preferred set of WSGI servers In-Reply-To: <42712537.2010805@colorstudy.com> References: <42712537.2010805@colorstudy.com> Message-ID: <1c466c6108a9b995f4ffcf0a829102ab@fuhm.net> On Apr 28, 2005, at 2:02 PM, Ian Bicking wrote: twisted.web2 supports: HTTP, HTTPS, CGI, and I wrote SCGI yesterday and will commit it this weekend. FastCGI looks like a complicated protocol, so it'll probably be a bit harder than SCGI to implement. Is there actually a reason to support it as well as SCGI? Right now, web2 runs WSGI apps in threads. It'd be possible to write a wrapper that runs them in their own processes, too, but would that give any advantage vs the simple CGI->WSGI wrapper that already exists? You can just use web2's ability to run CGIs to run WSGI-in-separate-processes also. It is my desire to enable twisted.web2 to be used to host any application in any environment, by providing a number of different deployment options and app running options. I've been developing it somewhat under the radar, but It's getting to the point where it's fairly stable. At this point the major TODO is sitting down and writing some documentation, so I'll try to get an alpha release out soon. Here's the high-level description from the (work-in-progress) docs: > Twisted.web2 is an asynchronous HTTP 1.1 server written for the > Twisted internet framework. It provides a RFC 2616 compliant HTTP 1.1 > protocol implementation, with pipelined and persistent request > support, in a non-blocking threadless manner. > > It also includes a simple web framework with request and response > objects, static file support, error handling, form upload support, > HTTP range support, pre-built parsers for all standard headers, and a > bunch of other goodies. > > It is deployable as a standard TCP or SSL web server, or as a CGI > script invoked by another server. Other deployment mechanisms (such as > SCGI) are planned. > > In addition to running native twisted.web2 applications, it can also > run any WSGI or CGI application, or, via compatibility wrappers, most > applications written for the older twisted.web and nevow APIs. > > Currently, twisted.web2 does not include a HTTP client or proxy, but > will at a future date. Also includes extra goodies such as automatic range support for all resources, automatic if-[un]modified-since/if-[none-]match support if you provide the proper headers, and optional automatic gzipping. James From steve at holdenweb.com Fri Apr 29 20:20:05 2005 From: steve at holdenweb.com (Steve Holden) Date: Fri Apr 29 20:20:04 2005 Subject: [Web-SIG] Re: Preferred set of WSGI servers In-Reply-To: <44860.66.192.34.8.1114797293.squirrel@66.192.34.8> References: <6654eac405042906187d99b6fd@mail.gmail.com> <4272668A.1060706@colorstudy.com> <42726E1A.6010809@holdenweb.com> <44860.66.192.34.8.1114797293.squirrel@66.192.34.8> Message-ID: <42727AD5.3050100@holdenweb.com> mike bayer wrote: > while this is not an exciting option, if separate per-user processes are > needed, there is always apache going via mod_proxy to sub-instances of > apache that run for each user's account on different ports, each running > mod_python. im not sure how feasable that is to run many servers in > various shared environments but I would think most other approaches > involve some per-user daemon as well...and i think apache is pretty > lightweight if configured so. > Well yes, that *could* be done, but I don't think many Apache hosting providers currently do it - most of them simply give each customer site its own virtual host, with appropriate AllowOverrides to allow the customers to control the site through .htaccess files. Another possibility for shared hosting is chroot jails, but again this is a minority of hosting providers. > >>>Peter Hunt wrote: >>> [no he didn't, that was me] >> >>Seems to me that mod_python isn't really suitable for a shared-hosting >>environment because of the need to restart the server when >>indirectly-imported modules are updated, a situation that I believe >>mod_python does not detect. > regards Steve -- Steve Holden +1 703 861 4237 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/ From jjinux at gmail.com Fri Apr 29 20:30:57 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri Apr 29 20:30:58 2005 Subject: [Web-SIG] Preferred set of WSGI servers In-Reply-To: <64ddb72c05042822267c0a0696@mail.gmail.com> References: <42712537.2010805@colorstudy.com> <64ddb72c05042822267c0a0696@mail.gmail.com> Message-ID: The reloading problem is a tough one, and Aquarium went through a lot of iterations before it came to a "good solution". Now days, I have a property which tells what types of modules I'm interested in reloading. Then, if any module is stale, I reload all of those modules. This takes care of cases of the parent being stale, requiring the child to be reloaded. Modules that aren't subject to reloading aren't reloaded because they probably have important state. I have very few of those, and they require a restart if they change. Fortunately, that is rare. I'm more than happy to share code. It's rather small. Best Regards, -jj On 4/28/05, Rene Dudfield wrote: > On 4/29/05, Ian Bicking wrote: > > Here's the full set of WSGI-enabled servers I'd like to see (and in > > some way encorporate into Paste, of course): > > > > * CGI gateway (done: not sure of canonical location) > > * Simple threaded HTTP server (done: wsgiutils) > > * mod_python (?) > > * ISAPI (done: http://isapi-wsgi.python-hosting.com/) > > * Zope product (not done) > > > > Those are all rather specific to certain environments. Then I'd like > > to see two other servers: multi-process and threaded. Obviously the > > two imply very different architectures. Well, maybe not, but it seems > > like it to me. > > > > Right now we have several servers with different purposes: SCGI (one > > for forking and threaded each), FastCGI, HTTP, etc. I don't really > > like that, I feel protocol support should be a separate concern. I > > would like for the two "preferred" deployment servers to support this > > set of protocols: > > > > * HTTP > > * HTTPS > > * FastCGI > > * SCGI > > > > Support for other protocols like FTP doesn't hurt, of course, but > > their exact relation to WSGI is unclear. Right now servers seem to be > > FastCGI-ish (which includes SCGI) *or* HTTP. I don't think this > > protocol choice should restrict the server choices. I think > > FastCGI-ish setups can be easier to configure an integrate; other > > people may disagree, but I don't see any technical reason not to > > include all these protocols, so why not make everyone happy? (Note > > that I'm personally not concerned about the fancier features of > > FastCGI specifically, like starting processes and doing pooling as > > part of the protocol; I'm only interested in the delegation of > > requests to different processes.) > > > > Twisted is an obvious contender for the threaded server. However, to > > my knowledge it does not support FastCGI or SCGI (as a client). Also, > > WSGI support is only available for twisted.web2, which doesn't have a > > release yet. Lastly, the installation isn't as easy as it could be -- > > it requires Zope interfaces, which includes C extensions. So, there's > > some work to be done there. To their credit, with the recent split of > > Twisted into smaller packages they are moving in the right direction. > > Medusa is a simpler alternative; it's not as actively maintained, but > > there aren't as many packaging or dependency issues. > > > > Twisted reverse proxied by apache2 is my server of choice at the > moment for wsgi. As it can nicely run multiple wsgi apps. I use > apache2 instead of lighttpd now, because it supports compression, and > caching of dynamic content. > > With twisted I can also do other things in the background quite > easily. Like making http requests, sending/serving email, etc etc. > > There are still a few bugs in twisted.web2 as it is not released. > However there are a few active contributors to web2 who are quick to > help out with any problems. > > Here is an example of using twisted to serve two wsgi apps with web2. > http://www.madecollective.com/~rene/two_wsgi.py > > > > For the forking server, I really have no idea. There's several out > > there, but I don't know about all the features that are desirable or > > implemented. Several of the servers don't speak HTTP. I can't make > > any specific criticisms, because I don't even know what to look at. > > > > http://www.saddi.com/software/py-lib/ with his flup package contains > experimental forking/preforking wsgi servers. Have you seen these? > > The above also contains an XMLRPC wsgi server. So you can easily > serve xmlrpc on with wsgi too :) > > > > Lingering in the background of these server discussions are issues > > like user isolation and module reloading (and/or server restarting). > > > > Thoughts, reactions? > > > > Another nice thing about twisted is that it has server restarting in > there. It also has module reloading. Allthough I don't use either. > > I think a separate process which monitors things and sends the app > restart, reload, signals is a good idea. Then in development you can > have the monitor send it a restart signal as soon as code/files > change. > > I use this bash script on debian linux to watch a directory and then > do stuff if any of my files change. It uses rsync, which is really > fast. It doubles as an automatic incremental backup service. With > putting files that have changed in separate time stamped directories. > It can be easily modified to save the backups on another machine too. > > watcher.sh > > #!/bin/bash > DIR_TO_WATCH=pretendpaper/pp > BACKUP_DIR=/tmp/ > CHANGED_DIR=/tmp/changed/ > export THE_DATE=`date +%Y_%m_%d_%H_%M_%s` > # you need to make sure the changed dir is writable, with good permissions. > # if you use /tmp uncomment these lines. > #mkdir -p $CHANGED_DIR > #chmod 755 $CHANGED_DIR > > NUM_CHANGED=`rsync --delete --backup > --backup-dir=$CHANGED_DIR/$THE_DATE --progress -v -a $DIR_TO_WATCH > $BACKUP_DIR | grep "100%" | wc -l` > > if [ "$NUM_CHANGED" = "0" ]; then > echo "nothing changed" > else > echo "something changed" > # Put your commands here. eg. rebuild cheetah templates, restart > server, flush caches etc. > fi > > Then I use `watch -n 2 ./watcher.sh` in an xterm. Which makes it > run every two seconds. > > User isolation: > ---------------------- > > Running different processes for different users is the best way I > think. That way you can use operating system level resource > constraints and auditing. On pretendpaper.com I have given users with > different roles separate processes. Each process also uses a > different database user, to add in another layer of security. So > basic members database user does not have access to > select/delete/update tables, the process serving their requests should > not be able to access them either. > > On linux(and other unixes) with ulimit you can set each group of users > having a different amount of available memory and cpu usage. eg > ulimit -v 50000 will limit it to running 50 megabytes of virtual > memory. > > Cheers, > > Rene Dudfield > > e: "@".join( reversed(['madecollective.com', 'rene']) ) > w: http://www.madecollective.com/ > w: http://www.pretendpaper.com/ > For creative, and technical services contact us @ made. > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/jjinux%40gmail.com > -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From ianb at colorstudy.com Fri Apr 29 20:35:12 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 29 20:37:48 2005 Subject: [Web-SIG] Preferred set of WSGI servers In-Reply-To: <1c466c6108a9b995f4ffcf0a829102ab@fuhm.net> References: <42712537.2010805@colorstudy.com> <1c466c6108a9b995f4ffcf0a829102ab@fuhm.net> Message-ID: <42727E60.6010607@colorstudy.com> James Y Knight wrote: > On Apr 28, 2005, at 2:02 PM, Ian Bicking wrote: > > > twisted.web2 supports: HTTP, HTTPS, CGI, and I wrote SCGI yesterday and > will commit it this weekend. FastCGI looks like a complicated protocol, > so it'll probably be a bit harder than SCGI to implement. Is there > actually a reason to support it as well as SCGI? While FastCGI isn't defined as having layers, I'd kind of treat it like that. From what I can tell, it's like SCGI with a stderr stream...? Other parts could probably be ignored. The advantage is that it's more widely supported/installed. I think, kind of. For example, lighthttpd has built-in FastCGI support. But then I bet it would be easy to convince them to include SCGI too. > Right now, web2 runs WSGI apps in threads. It'd be possible to write a > wrapper that runs them in their own processes, too, but would that give > any advantage vs the simple CGI->WSGI wrapper that already exists? You > can just use web2's ability to run CGIs to run > WSGI-in-separate-processes also. Well, the possible advantages: * Fork instead of spawn a process, so as to save startup time. Not possible on Windows. * Reuse processes, so as to save startup time. Possible on Windows, I think. Though they went through all of that trouble in Apache 2 (and even some weird fork of 1.3) to make it threaded, so maybe there's something I don't know. But I feel fuzzy on the details, so maybe some of this is a wash. Could you just run, say, ten non-threaded Twisted servers, and have the main server delegate a request via SCGI to one of those subservers? Is it acceptable for a Twisted server to block if you aren't serving concurrent requests? I don't do enough multiprocess programming to speak intelligently on some of this. > It is my desire to enable twisted.web2 to be used to host any > application in any environment, by providing a number of different > deployment options and app running options. > > I've been developing it somewhat under the radar, but It's getting to > the point where it's fairly stable. At this point the major TODO is > sitting down and writing some documentation, so I'll try to get an alpha > release out soon. One advantage of WSGI is that you don't need to document it. You'd still have to document server invocation and some other details, but lots of things people think of as part of the server needn't be (e.g., name-based virtual hosting). People who plan to do async programming under twisted.web2 probably need much more information about the internals, but WSGI users only need to know you Do The Right Thing (not how you do it). -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From jjinux at gmail.com Fri Apr 29 20:48:35 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri Apr 29 20:48:38 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> Message-ID: On 4/29/05, Greg Wilson wrote: > >>Greg Wilson wrote: > >>I think Ian Bicking or Michelle Levesque would be good choices for the > >>role... > > > mike bayer wrote: > > Michelle is currently a computer science undergrad at the university of > > Toronto. Are you saying all the people with many years of real-world > > experience building dynamic web applications and sites for the corporate > > world and the educational world should be forced to obey the technical > > direction of a college student with no professional experience ? > > Greg Wilson wrote: > Nope, I didn't say that at all (and I don't think that putting words in > people's mouths helps this discussion at all). This is open source: no > one can "force" anyone to do anything, not even Guido. I mentioned Ian > and Michelle because they know the players, have worked with most of the > candidate systems, and have demonstrated that they're willing to listen > as well as talk; there are undoubtedly other people who could "play > Guido" just as well. > > Thanks, > Greg > > p.s. Ka-Ping Yee was "just an undergrad" when he started contributing to > Python; I don't remember anyone thinking that ought to disqualify him. I have a lot of respect for Michelle, but she has a *long* way to before she can say she's worked with most of the candidate systems. There so *many* of them! Best Regards, -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From jjinux at gmail.com Fri Apr 29 21:04:02 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri Apr 29 21:04:04 2005 Subject: [Web-SIG] the world's gone crazy In-Reply-To: <427266D7.3070302@infrae.com> References: <427266D7.3070302@infrae.com> Message-ID: Hmm, I wasn't on this list at the time it happened. Was everyone going goofy when Mason came out? I'm sure Mason is still far more widely used than RoR. I've used Mason, and it's really nice. However, neither Mason nor RoR meet my needs because I'm a Python coder. My company, IronPort, has a million lines of Python code. Imagine my trying to introduce RoR to Sam Rushing, the author of Medusa. Not likely ;) Now, company-wide, we're using Aquarium. If I was working in Ruby, I'd use RoR. What's the big fuss? I couldn't care less as long as it's open source, although I do prefer Python ;) In the meantime, I'll continue stealing the interesting bits out of Mason, RoR, Zope, Struts (not often), etc. As soon as I write a WSGI adaptor, which is high on my TODO list, I can use that code too. What's to freak out over? Happy Hacking, -jj On 4/29/05, Martijn Faassen wrote: > Greg Wilson wrote: > > > Jeremy Hylton wrote: > >> I don't think a large web programming toolkit belongs in the Python > >> distribution. If anything, go the other way around and package a > >> particular version of Python with this web toolkit. > > > > > > Sure, both models have been successful in the past: PIL and Numeric are > > examples of "external, but only one" that have worked well. > > > > Perhaps someone who took part in the discussion about what XML tools to > > include in the core Python distro could chime in with a little history? > > The stuff started as an external package first, and still is an external > package (PyXML). In my opinion the whole XML in the core story sucks. > Not because the code in the core is bad, but because there's another > package, PyXML, that completely overrides it in a non-bug-compatible > way, meaning you now need to test your code that uses the XML core-code > both with and without (various versions of..) PyXML installed. If this > is to be done for any hypothetical web framework, let's please not do it > the XML way... This is not due to a mistake; at the time the core was > deliberately made to look for PyXML. > > There are a ton of non-core XML frameworks around for Python, enjoying > considerate popularity. The python 'xml' package is not the "one true > way" to do XML with Python, and certainly doesn't enjoy anything near > the popularity and buzz of Ruby on Rails, say. I don't see why the > situation for any higher-level web framework should be different in this > respect. > > >> I'm also skeptical of a plan that sets out to build the one right way > >> that everyone will use. > > > > Well, then it's a good thing that's not what I'm asking for, isn't it? > > ;-) I think there's a place for entry-level (i.e., smaller than Zope) > > web app frameworks; the current confusion is about as helpful as having > > eight competing regular expression libraries, or six different > > "standards" for connecting to databases. Appointing a BDFWFOY is one > > way to solve the problem (though admittedly less popular than denying it > > exists ;-). I'd welcome others... > > What is exactly problem? That Ruby on Rails is taking mindshare away > from Python? And so's PHP, and ASP, and Java, and so is Python from > Java, and Zope from PHP, and so on. Could you explain why you consider > Ruby on Rails to be different? Evidently you are of this opinion. > > I do see a problem with Python web frameworks, but it's got little to do > with Ruby on Rails, though I do think we can learn from its marketing > success. My problem is that of islands; people are often sitting in the > island of their own framework and making it hard for people on their > islands to work together. I think there's far more code we can share > between frameworks than we've been doing in the past. More sharing code > and integrate various bits using standard ways is good as it allows us > to join forces and not waste time reinventing wheels. > > I think encouraging sharing, and setting up the circumstances to make > sharing more easy (such as WSGI) is the only realistic way to reduce the > confusion between Python web frameworks and make them start to evolve > towards each other, which should also help marketing and reduce > developer confusion during uptake. > > In the Zope world, we've got islands too, in the form of CMSes built on > top of Zope, such as Plone, CPS and Silva. We're currently in the > process of trying to get off our islands and converge, helped by the > transition to Zope 3 component technologies. I've been helping with that > process, and it's fairly hopeful. So now I've started to set my sights > a bit further and I'm trying to help Zope itself become less of an > island within the Python world. > > Regards, > > Martijn > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/jjinux%40gmail.com > -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From jjinux at gmail.com Fri Apr 29 21:28:47 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri Apr 29 21:28:51 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <42726BB3.7000500@colorstudy.com> References: <427266D7.3070302@infrae.com> <42726BB3.7000500@colorstudy.com> Message-ID: At the last Bay Piggies meeting, as well as at PyCon, Guido felt strongly that no Python Web application framework belonged in the Python standard library for these reasons: o The release schedule for such a library doesn't match the release schedule of Python. Imagine having to wait a year before an update to the library came out? That'd be unacceptible. o Much of this is a matter of taste. o There's no clear cut winner. Zope and Twisted aren't a part of Python either, despite the fact that they're wildly successful. Perhaps it just doesn't make sense. Best Regards, -jj On 4/29/05, Ian Bicking wrote: > Martijn Faassen wrote: > > There are a ton of non-core XML frameworks around for Python, enjoying > > considerate popularity. The python 'xml' package is not the "one true > > way" to do XML with Python, and certainly doesn't enjoy anything near > > the popularity and buzz of Ruby on Rails, say. I don't see why the > > situation for any higher-level web framework should be different in this > > respect. > > I think the standard library is difficult, because it mixes all sorts of > things together. I involves methodology -- very conservative, backward > compatible, slow to improve -- which suggests that a package should be > at a certain point in its life. It also suggests applicability, that > the package should be reasonably generic and not unduly complex. And it > encourages people to use it, either directly or indirectly, instead of > alternatives. All of these are important, but they are fairly separate. > mxDateTime would have been a very capable implementation of date-time > objects, and was a de facto standard, but because of methodology > conflicts that didn't happen (I assume because mxDateTime had to be > released on a schedule determined by commercial concerns). Wx is > similar but even worse; there's no practical way I can imagine it going > into the standard library, and it's not just inertia, but rather Tk > actually *remains* ideal simply because it is largely dead. > > Generally the standard library is become much less practical way to move > Python forward. Backporting has to be extensive for the library to be > viable to use. Up-front design becomes burdensome -- I'm sure there was > a lot of useful things that came out of the decimal discussion, but the > whole process seemed odd and drawn-out from the outside. Copied designs > can shortcut this (e.g., logging), but lead to other design problems. > And even as this happens, distributions are often pulling the standard > library apart into pieces. Batteries Included has certainly helped me > as a developer, but it's not what I have my eyes on for the future. > > It would be nice if process and politics could be separated. There's a > lot of modules in the standard library that no one should use, even if > they aren't marked "deprecated" -- there might be no reason for those > modules to go away ever, but they just shouldn't get new users. > Similarly there's modules not in the standard library that should be de > facto standards, but for good reason can't be part of the standard > library development process. But there's no way to suggest that to > people except for information question-and-answer sessions in IRC or > mailing lists, or the the vague and unpredictable rankings of Google. > > "Politics" might be a bad name for this; it doesn't have to be > contentious -- for instance, sgmllib has no hero who will be offended > that people are steered away from it. But I can't think of a better > name. Anyway, I think it might be possible to resolve that issue more > easily than the technical issues involved in extending the standard library. > > -- > Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/jjinux%40gmail.com > -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From jjinux at gmail.com Fri Apr 29 21:50:01 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri Apr 29 21:50:04 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> Message-ID: When I came to IronPort, I had to act as such a benevolent dictator, or rather, a benevolent concensus builder. I worked with Sam Rushing, of Medusa fame; Paul Clegg who worked on ClearSilver; Eric Huss who had his own templating language (what good Python programmer doesn't!); my boss who was from the Struts world and who favored Cheetah and WebWare; I had written Aquarium and enjoyed Mason. We chose Aquarium mainly because it was thread agnostic, and could be easily run within a co-routine, stackless Python environment. A lot of changes were made to Aquarium to incorporate what the other guys thought were key parts of the other frameworks. We chose Cheetah because it was stable, and because I personally don't like templating engines that act like handcuffs. Note, I'm not trying to force Aquarium on *anybody*. I wrote it because I needed it. I open sourced it because I like sharing. My personal feeling is that having such a benevolent dictator would probably be a bad thing. Concensus is a good thing. Perhaps the way the IETF works is a good model. There's a couple chairmen from different companies who mostly sit back and watch as people fight it out. On the other hand, concensus in the IETF can take years. Furthermore, it does nothing to reduce duplicated code. The IETF just makes sure that different systems can play nicely with each other. Is it just me or does the Python Web application framework world seem about as obstinate a problem as the Palestinian mess? ;) Happy Hacking :-D -jj On 4/29/05, Greg Wilson wrote: > Hi JJ; thanks for your message. > > > I have a lot of respect for Michelle, but she has a *long* way to > > before she can say she's worked with most of the candidate systems. > > There so *many* of them! > > Best Regards, > > -jj > > It was just a name --- I notice no one has commented on me suggesting Ian > Bicking (hi, Ian ;-), and I'm sure there are others. I think it would be > important to pick someone who isn't the author of any particular > framework, though; maybe one of the creators of Trac? > > Thanks, > Greg > > p.s. Trac's a perfect example of why I think we need to pick one framework > (or one combination of tools, if you prefer) and put our effort into that. > It's a classic MVC web app, but its deadline-driven authors decided they'd > be better off "rolling their own" than using WebWare, CherryPy, Twisted, > etc. We'll know we've solved this problem when the *next* group to come > along can get their app up and running in half the time, with half as many > separate installs to coordinate. > -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From jjinux at gmail.com Fri Apr 29 22:05:28 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri Apr 29 22:05:35 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> Message-ID: At the risk of offending you guys with my continual blabbering, I do like the "tools not policy" approach shared by the FreeBSD and Mason worlds. For instance, the Apache project has nice session libraries for Perl. Everyone wrote their own plugins for the common API. Since the library is a tool, not a framework, anyone can use it with their existing framework. At the risk of showing my ignorance concerning WSGI, it's easier to use a session library in Perl than a session library in WSGI, because the session library has no knowledge of anything in your application, not even a context object. It doesn't even have to be a Web application. It doesn't tell you how you have to write your app. You pick the version of the session library that makes sense for your environment. At the risk of sounding overzealous, maybe we should just steal their API. We can each rewrite whatever session implementations we need, stealing heavily from our existing code. For something so low level, this is one place were we could share easily, as I've heard many other people say. It'd be easy to use it with WSGI, but wouldn't necessarily even be Web related :-/ Happy Hacking, -jj -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From todd at slack.net Fri Apr 29 22:22:07 2005 From: todd at slack.net (Todd Grimason) Date: Fri Apr 29 22:16:39 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> Message-ID: <20050429202207.GA26081@detroit.slack.net> * Shannon -jj Behrens [2005-04-29 15:55]: > Is it just me or does the Python Web application framework world seem > about as obstinate a problem as the Palestinian mess? ;) ack! no real world problems please. i might be the only one here who hasn't written my own framework, but as a sample customer, i'm coming to the conclusion that what's lacking most on any of the number of existing, good frameworks is just "spit & polish". In other words, good docs, good tutorials, sample applications (beyond 10-liners), and yes, as much as many coders seem to distain it, good-looking websites. If someone coming to web programming from X or Y language (X or Y not being python or ruby), and looks at the Rails site compared to almost any of the python frameworks, they'd very likely conclude it [Rails] is a more mature, widely used, and professional toolkit -- even though in many cases that's completely not true. There is probably more sample code (test apps, example sites) on the Rails site in teh past 6 months than a bunch of the python kits all combined over years. And for a lot of people, "you can go ahead and just plug in whatever you like for the backend or templates at this point" isn't enough instruction or guidance. I agree maintaining the flexibility is important, but even better is showing at least one common way of tying things together. And whenever possible, getting users of a framework to publicly admit it helps - witness the attention on python by Google, NASA, et. al admitting to their usage, as well as Rails' drawing attention with Basecamp and 43things (regardless of your opinions of those, they draw attention). Of course not all companies are willing to publicly admit to things... though jj did spill the beans on his company ;-) That should be on the Aquarium site! Of course these are broad sweeping statements but I think they're fairly accurate. Now how to get any of the projects more people and polish and docs... well I guess that's the rub. But in summary what's needed *is* "marketing" to expand awareness and usage of the good frameworks already existing in python. A common platform on Paste will only make things even better - but at this point the lack of Paste isn't the main hinderance. I hope this point of view is somewhat helpful and not just annoying(!) -- ______________________________ toddgrimason*todd-AT-slack.net From renesd at gmail.com Fri Apr 29 22:41:45 2005 From: renesd at gmail.com (Rene Dudfield) Date: Fri Apr 29 22:41:47 2005 Subject: [Web-SIG] Preferred set of WSGI servers In-Reply-To: <1c466c6108a9b995f4ffcf0a829102ab@fuhm.net> References: <42712537.2010805@colorstudy.com> <1c466c6108a9b995f4ffcf0a829102ab@fuhm.net> Message-ID: <64ddb72c05042913416764384a@mail.gmail.com> Supporting fastcgi is useful for two reasons. Supporting fastcgi so that you can be called by another httpd, eg apache, or lighttpd. Some people allready use fastcgi to call php, python or ruby on rails apps. Supporting it to call php, or ruby on rails apps. This way, twisted could run .php files as almost fast as apache. lighttpd an async httpd server uses fastcgi for running php, and it is quite fast, as well as being more secure than running mod_php with apache. On 4/30/05, James Y Knight wrote: > On Apr 28, 2005, at 2:02 PM, Ian Bicking wrote: > > > twisted.web2 supports: HTTP, HTTPS, CGI, and I wrote SCGI yesterday and > will commit it this weekend. FastCGI looks like a complicated protocol, > so it'll probably be a bit harder than SCGI to implement. Is there > actually a reason to support it as well as SCGI? > > Right now, web2 runs WSGI apps in threads. It'd be possible to write a > wrapper that runs them in their own processes, too, but would that give > any advantage vs the simple CGI->WSGI wrapper that already exists? You > can just use web2's ability to run CGIs to run > WSGI-in-separate-processes also. > > It is my desire to enable twisted.web2 to be used to host any > application in any environment, by providing a number of different > deployment options and app running options. > > I've been developing it somewhat under the radar, but It's getting to > the point where it's fairly stable. At this point the major TODO is > sitting down and writing some documentation, so I'll try to get an > alpha release out soon. > > Here's the high-level description from the (work-in-progress) docs: > > Twisted.web2 is an asynchronous HTTP 1.1 server written for the > > Twisted internet framework. It provides a RFC 2616 compliant HTTP 1.1 > > protocol implementation, with pipelined and persistent request > > support, in a non-blocking threadless manner. > > > > It also includes a simple web framework with request and response > > objects, static file support, error handling, form upload support, > > HTTP range support, pre-built parsers for all standard headers, and a > > bunch of other goodies. > > > > It is deployable as a standard TCP or SSL web server, or as a CGI > > script invoked by another server. Other deployment mechanisms (such as > > SCGI) are planned. > > > > In addition to running native twisted.web2 applications, it can also > > run any WSGI or CGI application, or, via compatibility wrappers, most > > applications written for the older twisted.web and nevow APIs. > > > > Currently, twisted.web2 does not include a HTTP client or proxy, but > > will at a future date. > > Also includes extra goodies such as automatic range support for all > resources, automatic if-[un]modified-since/if-[none-]match support if > you provide the proper headers, and optional automatic gzipping. > > James > > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/renesd%40gmail.com > From renesd at gmail.com Fri Apr 29 22:52:01 2005 From: renesd at gmail.com (Rene Dudfield) Date: Fri Apr 29 22:52:04 2005 Subject: [Web-SIG] Re: Preferred set of WSGI servers In-Reply-To: <42726E1A.6010809@holdenweb.com> References: <6654eac405042906187d99b6fd@mail.gmail.com> <4272668A.1060706@colorstudy.com> <42726E1A.6010809@holdenweb.com> Message-ID: <64ddb72c05042913523dd4e3de@mail.gmail.com> On 4/30/05, Steve Holden wrote: > Ian Bicking wrote: > > Peter Hunt wrote: > > > >> Actually, I think a mod_wsgi for Apache and IIS would be one of _the_ > >> most important things for WSGI. I think that it would search for a > >> __wsgi__.py file (or maybe something with a better name) which would > >> expose a WSGI application named "application" that would handle > >> requests for the directory in which it exists. This would allow a user > >> to simply drop a file on the webserver and have it work. It would be > >> excellent for shared hosting solutions. > > > > > > Would it? From what I can tell, mod_php is very popular in shared > > hosts, but neither mod_python nor mod_perl are. I don't think mod_* > > makes it necessarily host-friendly. But from what I can tell of > > mod_python, it would be relatively easy to have something like .wsgi > > files, and give them a handler that runs them as WSGI apps. > > > > I can imagine a good setup for hosts being one based on forking per-user > > processes, which is adaptive primarily to scale down -- e.g., a largely > > dorman app could have 1 or even 0 processes running (at 0 it becomes > > similar to CGI, but presumably the process would stay around for some > > time to respond to subsequent requests). The "scaling down" scenario I > > often think about would be a email contact form -- one of those things > > that has to be an app, can be implemented and deployed separately from > > other aspects of the site, and yet it's clear waste of resources to keep > > a process always around to respond to such requests. Though it's > > actually someplace where CGI would work just fine; but lets say you > > don't want to educate the developer about when they might want to use > > other deployment strategies (which is a rather complex discussion > > really, that would be better avoided by providing one really good > > strategy and telling everyone to use it). > > > > Maybe mod_python is appropriate for that kind of situation, I'm not > > sure. It could also be implemented with a second, single process that > > was developed and deployed separately from Apache. > > > > As for IIS and ISAPI, I can't say I really know what the archicture of > > that system is, so I don't know. Oh, but that does remind me of another > > WSGI server: ASP. Though I remember vaguely this came up before and it > > seemed infeasible because ASP didn't support anything like PATH_INFO. Oh > > well, I can't remember now. > > > Seems to me that mod_python isn't really suitable for a shared-hosting > environment because of the need to restart the server when > indirectly-imported modules are updated, a situation that I believe > mod_python does not detect. > I think this is changed since a recent python2.4? Not sure if it also affects mod_python... from the release notes 'What's New in Python 2.4?': http://www.python.org/2.4.1/NEWS.html """When importing a module M raises an exception, Python no longer leaves M in sys.modules. Before 2.4a2 it did, and a subsequent import of M would succeed, picking up a module object from sys.modules reflecting as much of the initialization of M as completed before the exception was raised. Subsequent imports got no indication that M was in a partially- initialized state, and the importers could get into arbitrarily bad trouble as a result (the M they got was in an unintended state, arbitrarily far removed from M's author's intent). Now subsequent imports of M will continue raising exceptions (but if, for example, the source code for M is edited between import attempts, then perhaps later attempts will succeed, or raise a different exception). This can break existing code, but in such cases the code was probably working before by accident. In the Python source, the only case of breakage discovered was in a test accidentally relying on a damaged module remaining in sys.modules. Cases are also known where tests deliberately provoking import errors remove damaged modules from sys.modules themselves, and such tests will break now if they do an unconditional del sys.modules[M].""" Cheers, Rene Dudfield e: "@".join( reversed(['madecollective.com', 'rene']) ) w: http://www.madecollective.com/ w: http://www.pretendpaper.com/ For creative, and technical services contact us @ made. From pje at telecommunity.com Fri Apr 29 23:05:54 2005 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri Apr 29 23:02:54 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> Message-ID: <5.1.1.6.0.20050429170022.0319eec0@mail.telecommunity.com> At 01:05 PM 4/29/05 -0700, Shannon -jj Behrens wrote: >At the risk of showing my ignorance concerning >WSGI, it's easier to use a session library in Perl than a session >library in WSGI, because the session library has no knowledge of >anything in your application, not even a context object. I don't understand why you think WSGI requires anything different. Actually, most of the time I wonder why people think sessions should be implemented with WSGI middleware at all, as opposed to just using a library. Each time, however, I eventually figure out that it's because they'd like responses affecting the session (e.g. Set-Cookie headers) to be done automatically, without the app needing to care about it. Of course, management of the "session state" vs. management of the "session content" are largely orthogonal concerns that might be worth standardizing an interface between, so that you can mix and match session-state mechanisms (cookies, subdomains, paths, query string tags) with session-content mechanisms (files, database, shared memory, etc.). From jjinux at gmail.com Fri Apr 29 23:05:46 2005 From: jjinux at gmail.com (Shannon -jj Behrens) Date: Fri Apr 29 23:05:49 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: <5.1.1.6.0.20050429170022.0319eec0@mail.telecommunity.com> References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> <5.1.1.6.0.20050429170022.0319eec0@mail.telecommunity.com> Message-ID: I think we're violently agreeing. -jj On 4/29/05, Phillip J. Eby wrote: > At 01:05 PM 4/29/05 -0700, Shannon -jj Behrens wrote: > >At the risk of showing my ignorance concerning > >WSGI, it's easier to use a session library in Perl than a session > >library in WSGI, because the session library has no knowledge of > >anything in your application, not even a context object. > > I don't understand why you think WSGI requires anything different. > > Actually, most of the time I wonder why people think sessions should be > implemented with WSGI middleware at all, as opposed to just using a > library. Each time, however, I eventually figure out that it's because > they'd like responses affecting the session (e.g. Set-Cookie headers) to be > done automatically, without the app needing to care about it. > > Of course, management of the "session state" vs. management of the "session > content" are largely orthogonal concerns that might be worth standardizing > an interface between, so that you can mix and match session-state > mechanisms (cookies, subdomains, paths, query string tags) with > session-content mechanisms (files, database, shared memory, etc.). > > -- I have decided to switch to Gmail, but messages to my Yahoo account will still get through. From sa at c-area.ch Fri Apr 29 23:24:18 2005 From: sa at c-area.ch (Steven Armstrong) Date: Fri Apr 29 23:24:22 2005 Subject: [Web-SIG] Preferred set of WSGI servers In-Reply-To: References: <42712537.2010805@colorstudy.com> Message-ID: <4272A602.80504@c-area.ch> On 04/29/05 03:38, Mark Rees wrote: > On 4/29/05, Ian Bicking wrote: >> Here's the full set of WSGI-enabled servers I'd like to see (and in >> some way encorporate into Paste, of course): >> >> * CGI gateway (done: not sure of canonical location) >> * Simple threaded HTTP server (done: wsgiutils) >> * mod_python (?) > There is http://www.c-area.ch/code/mp_wsgi_handler.py but the licence > may be an issue. It is based on this email from the list: > > http://mail.python.org/pipermail/web-sig/2004-October/000980.html > Changed the licence to MIT. It was just a quick hack to get Pyblosxom to play nicely with mod_python, but has so far worked well & stable with all wsgi apps I've used. Feel free to edit/change/update/whatever. If anybody makes any improvements I'ld be happy to get a patch. cheers Steven > >> * ISAPI (done: http://isapi-wsgi.python-hosting.com/) > Currently looking at best way of getting isapi-wsgi to work with paste. > > Regards > > Mark > _______________________________________________ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: http://mail.python.org/mailman/options/web-sig/sa%40c-area.ch From ianb at colorstudy.com Fri Apr 29 23:27:52 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri Apr 29 23:30:48 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> Message-ID: <4272A6D8.8050009@colorstudy.com> Shannon -jj Behrens wrote: > At the risk of offending you guys with my continual blabbering, I do > like the "tools not policy" approach shared by the FreeBSD and Mason > worlds. This is exactly what I see WSGI accomplishing. It's too lame of an interface to be anyone's policy (in a good way, of course). It's always been possible to use external libraries, but it's difficult to use libraries that participate in the request/response cycle, and WSGI makes that possible. This can be used in both mundane and interesting ways. I think sessions are a bit mundane, but they work well, allowing the session "library" to modify both request and response, providing an isolated and well-scoped way to add session services to a WSGI application. This doesn't preclude such a library (middleware) relying on other libraries for the bulk of their work (e.g., persistence); indeed, I think it's best to keep middleware as small as possible. A more interesting example of this kind of reuse is URL parsing. Typically URL parsing is one of those intimate details of a framework that can't be factored out. But paste.urlparser.URLParser is fairly successful, I think, at implementing a specific algorithm without greatly effecting other parts of the system; it doesn't make any assumptions about the applications it delegates to, or about the environment it is run in. It's not the only URL parser you'd want -- but that's a big part of what makes it interesting; it separates one kind of policy (URL resolution) from another (how resources are expressed). I think other people have worried that this sort of thing is all fun and games, but it doesn't give us a concise and unified framework. But by firmly keeping policy either decoupled (urlparser) or entirely removed (session middleware) it makes it possible to create a framework that is the amalgam of these pieces of code without sacrificing internal consistency. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From robert.leftwich at gmail.com Sat Apr 30 02:24:17 2005 From: robert.leftwich at gmail.com (Robert Leftwich) Date: Sat Apr 30 02:23:48 2005 Subject: [Web-SIG] Re: Just lost another one to Rails In-Reply-To: References: <57015.66.192.34.8.1114784644.squirrel@66.192.34.8> Message-ID: <4272D031.8090703@gmail.com> Shannon -jj Behrens wrote: > When I came to IronPort, I had to act as such a benevolent dictator, > or rather, a benevolent concensus builder...Note, I'm not trying to force > Aquarium on *anybody*. I wrote it because I needed it. I open > sourced it because I like sharing. > To add to JJ's background on Aquarium, note that IronPort's backing not only includes a commitment to using Aquarium in at least 7 different divisions of the company but also paying him to work on Aquarium part time. In conversations with JJ, I learnt that this commitment did not come lightly, but instead as a result of an internal comparison of the leading Python web frameworks. The comparison was biased a little against Aquarium by having the initial thought that it would/could not be the best solution and given the calibre of the people at IronPort and the quality of the competition the result is no mean feat. Recently, I've implemented fundamentally the same web application in Quixote, Subway and Aquarium and found that I much prefer to work with Aquarium. One major reason for this is the 'tools, not policy' approach that Aquarium takes. Another plus is that it leverages the full power of Cheetah, handling the issue of compilation of templates totally seamlessly. This eliminates any manual recompilation as well as the need to continually stop and restart the server, which, while not a huge problem, is surprisingly beneficial to the 'flow' of hacking away at a solution. The mailing list support from all projects has been fantastic, but I must highlight the dedication of JJ, making himself available on a number of occasions well into the early hours of the morning answering questions and debugging issues that were uncovered when running Aquarium in a Windows environment. I strongly recommend that people take a serious look at the Aquarium project and while it may not have the same prescriptive nature and the depth of backup material as RoR (yet) it is a good place to start a search for the right tool for the job and for a project to become involved in. Robert PS I'm newly subscribed to the list so I apologise if, in my enthusiasm, I'm a little too 'project specific'. From steve at holdenweb.com Sat Apr 30 15:14:18 2005 From: steve at holdenweb.com (Steve Holden) Date: Sat Apr 30 15:14:17 2005 Subject: [Web-SIG] Re: Preferred set of WSGI servers In-Reply-To: <64ddb72c05042913523dd4e3de@mail.gmail.com> References: <6654eac405042906187d99b6fd@mail.gmail.com> <4272668A.1060706@colorstudy.com> <42726E1A.6010809@holdenweb.com> <64ddb72c05042913523dd4e3de@mail.gmail.com> Message-ID: <427384AA.6010207@holdenweb.com> Rene Dudfield wrote: > On 4/30/05, Steve Holden wrote: > [snippety snip] >> >>Seems to me that mod_python isn't really suitable for a shared-hosting >>environment because of the need to restart the server when >>indirectly-imported modules are updated, a situation that I believe >>mod_python does not detect. >> > > > > I think this is changed since a recent python2.4? Not sure if it also > affects mod_python... > > > from the release notes 'What's New in Python 2.4?': > http://www.python.org/2.4.1/NEWS.html > > """When importing a module M raises an exception, Python no longer > leaves M in sys.modules. Before 2.4a2 it did, and a subsequent import > of M would succeed, picking up a module object from sys.modules > reflecting as much of the initialization of M as completed before the > exception was raised. Subsequent imports got no indication that M was > in a partially- initialized state, and the importers could get into > arbitrarily bad trouble as a result (the M they got was in an > unintended state, arbitrarily far removed from M's author's intent). > Now subsequent imports of M will continue raising exceptions (but if, > for example, the source code for M is edited between import attempts, > then perhaps later attempts will succeed, or raise a different > exception). > > This can break existing code, but in such cases the code was probably > working before by accident. In the Python source, the only case of > breakage discovered was in a test accidentally relying on a damaged > module remaining in sys.modules. Cases are also known where tests > deliberately provoking import errors remove damaged modules from > sys.modules themselves, and such tests will break now if they do an > unconditional del sys.modules[M].""" > > This doesn't affect the situation I was talking about, which has nothing to do with import errors. mod_python will reload one of its application modules if the code is updated on disk - it checks the source files, IIRC, to verify they are still valid. However, if that module imports another, and that other module is changed, mod_python doesn't "follow the chain" to reload the imported module. Of course, for pure-Python modules you could arrange to have a page that reloaded all relevant modules, but this is effectively working around the framework rather than using it, it requires separate management, it won't work for extension modules (which can't be reloaded) and it may well leave incompatible objects from previous module versions lying around to cause trouble. regards Steve -- Steve Holden +1 703 861 4237 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/