[Web-SIG] more comments on Paste Deploy

[Ian Bicking]

Jim Fulton wrote:
>> A couple years back, I started writing a library to parse a more  
>> sophisticated, Python-like syntax to do the same sorts of things,  
>> but only got as far as the parser.
> 
> A few years back, we created a library to parse more sophisticated  
> apache-like syntax and I wish we hadn't.  The ini/config format is  
> pretty standard and, IMO, really quite adequate.  I'm convinced that  
> we don't really need another configuration format, at least not at  
> this level.

Details of the structure aside, I've found string:string dictionaries 
entirely sufficient for expressing every configuration I've wanted to 
do.  I'm very happy that Paste Deploy doesn't support Python syntax for 
anything.

>> that could get stdlib support and ultimately hosting company  
>> support.  This would actually give us a leg up on even PHP for ease- 
>> of-deployment.
> 
> Aside from the universal configuration file issue, I think this would  
> be a terrific thing for us to focus on.  Something I hear a lot is  
> how much easier PHP applications are to deploy to hosting providers.   
> I would *love* it is Python had a similar story, even if only for  
> smaller applications.
> 
> I'd love to get some input who know a lot about what makes deploying  
> PHP apps so easy.

Well, it's a big help that PHP doesn't have Python's import system.  Oh 
how I hate Python imports... anyway, since it just uses the filesystem 
everything is kind of naturally hierarchical and isolated.  There are 
some system-wide configurations (in php.ini) -- these cause deployers a 
lot of pain.  But they are mostly overridable with .htaccess, I think. 
Also there's not many libraries, and what libraries there are are 
typically shipped with the applications.  PEAR (the PHP library system) 
started after I stopped doing much of any PHP, so I don't know how it 
effects things.

PHP also gets a lot of benefit from a CGI-like execution model.  There's 
a ton of crap that gets swept under the rug by this -- lots of memory 
leaks, for instance.  As they've been building up larger frameworks 
built from PHP code, the CGI-like execution speed has also been hitting 
them.  But since they have a fairly large library written in C (that is 
persistent and shared) it's usually pretty reasonable; it's just when 
they tried to copy Rails that it started really biting them.

I think the database drivers are a bit of a red herring.  What 
extensions PHP has been compiled with is pretty fixed by the hosting 
provider -- they just happen to all provide database drivers for the 
databases they support.  Which is kind of a no-brainer; if they *cared* 
about Python they'd easily be able to do the same for Python.  It would 
help if Python shipped with one or two, but eh.

Anyway, my feelings are that it's: (a) simple hierarchy through the 
filesystem (which will make Chad all excited ;), (b) reliability of the 
CGI model, and (c) hosting providers give a damn.  We can't do much 
about (c).  (a) requires an isolation tool, but we have a few now.  (b) 
still needs doing.

That Python is theoretically faster than PHP due to its typical 
execution model doesn't mean much to hosting providers.  They tend to be 
memory-constrained more than CPU constrained anyway.  And if you have 
slow code, you personally suffer -- but if you use lots of memory, you 
make everyone suffer.  One thing many hosts do is just periodically kill 
user's processes if they hang around too long.  Most don't seem to care 
if you have long-running processes, though I've heard a few might 
disable your account.

Someone (but I've forgotten who) suggested a technique to assist with 
this.  The SCGI package has a script cgi2scgi, just a simple CGI script 
written in C that sends the request to another server; I think just a 
port, but I'm sure it could be extended easily enough to send it to a 
named socket.  Anyway, if there was just a bit of process management 
code in that script it could also serve as a launcher, doing on-demand 
launching of a server (Flup I suppose) and then passing it on to that 
script.  FastCGI does all these things, but setup can be fairly 
complicated and many implementations are buggy.  Anyway, extending 
cgi2scgi to do this, along with some isolated environment, should be a 
fairly simple way to make Python hosting on commodity hosts a lot easier.

Some of the hosts only give FTP access, and may not have a compiler.  So 
ideally you could assemble everything on your workstation and upload it 
in batch.  Probably a single Linux executable would be fine -- FreeBSD 
should be able to run it fine, and everything that matters (for this use 
case) is Linux or FreeBSD.  Hopefully Sidnei won't mind that we leave 
Windows out ;) -- commodity Windows hosting is another situation 
entirely (about which I know nothing).

-- 
Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org