[Web-SIG] A Python Web Application Package and Format

[Ian Bicking]

While we are focusing on points of contention, there may be more points of
consensus, but we aren't talking about those.

So, some initial thoughts:

While initially reluctant to use zip files, after further discussion and
thought they seem fine to me, so long as any tool that takes a zip file can
also take a directory.  The reverse might not be true -- for instance, I'd
like a way to install or update a library for (and *inside*) an application,
but I doubt I would make pip rewrite zip files to do this ;)  But it could
certainly work on directories.  Supporting both isn't a big deal except that
you can't do symlinks in a zip file.

I don't think we're talking about something like a buildout recipe.  Well,
Eric kind of brought something like that up... but otherwise I think the
consensus is in that direction.  So specifically if you need something like
lxml the application specifies that somehow, but doesn't specify *how* that
library is acquired.  There is some disagreement on whether this is
generally true, or only true for libraries that are not portable.

Something like a database takes this a bit further.  We haven't really
discussed it, but I think this is where it gets interesting.  Silver Lining
has one model for this.  The general rule in Silver Lining is that you can't
have anything with persistence without asking for it as a service, including
an area to write files (except temporary files?)  I assume everyone agrees
that an application can't write to its own files (but of course it could
execfile something in another location).

I suspect there's some disagreement about how the Python environment gets
setup, specifically sys.path and any other application-specific
customizations (e.g., I've set environ['DJANGO_SETTINGS_MODULE'] in
silvercustomize.py, and find it helpful).  Describing the scope of this, it
seems kind of boring.  In, for example, App Engine you do all your setup in
your runner -- I find this deeply annoying because it makes the runner the
only entry point, and thus makes testing, scripts, etc. hard.

We would start with just WSGI.  Other things could follow, but I don't see
any reason to worry about that now.  Maybe we should just punt on aggregate
applications now too.  I don't feel like there's anything we would do that
would prevent other kinds of runtime models (besides the starting point,
container-controlled WSGI), and the places to add support for new things are
obvious enough (e.g., something like Silver Lining's platform setting).  I
would define a server with accompanying daemon processes as an "aggregate".

An important distinction to make, I believe, is application concerns and
deployment concerns.  For instance, what you do with logging is a deployment
concern.  Generating logging messages is of course an application concern.
In practice these are often conflated, especially in the case of bespoke
applications where the only person deploying the application is the person
(or team) developing the application.  It shouldn't be *annoying* for these
users, though.  Maybe it makes sense for people to be able to include
tool-specific default settings in an application -- things that could be
overridden, but especially for the case when the application is not widely
reused it could be useful.  (An example where Silver Lining gets is all
backwards is I created a [production] section in app.ini when the very
concept of "production" is not meaningful in that context -- but these kind
of named profiles would make sense for actual application deployment
tools.)  An example of a setting currently in Silver Lining/app.ini that
should become a tool-specific default setting would be "default_location"
(the default place to upload your app to when you do "silver update").


There's actually a kind of layered way of thinking of this:

1. The first, maybe most important part, is how you get a proper Python
environment.  That includes sys.path of course, with all the accompanying
libraries, but it also includes environment description.  In Silver Lining
there's two stages -- first, set some environmental variables (both general
ones like $SILVER_CANONICAL_HOST and service-specific ones like
$CONFIG_MYSQL_DBNAME), then get sys.path proper, then import silvercustomize
by which an environment can do any more customization it wants (e.g., set
$DJANGO_SETTINGS_MODULE)
2. Define some basic generic metadata.  "app_name" being the most obvious
one.
3. Define how to get the WSGI app.  This is WSGI specific, but (1) is *not*
WSGI specific (it's only Python specific, and would apply well to other
platforms)
4. Define some *web specific* metadata, like static files to serve.  This
isn't necessarily WSGI or even Python specific (not that we should bend
backwards to be agnostic -- but in practice I think we'd have to bend
backwards to make it Python-specific).
5. Define some lifecycle metadata, like update_fetch.  These are generally
commands to invoke.  IMHO these can be ad hoc, but exist in the scope of (1)
and a full "environment".  So it's not radically different than anything
else the app does, it's just we declare specific times these actions
happen.  URL fetching and script running are both fine, because we start at
(1) and not (3) (this is in contrast to App Engine, which only defines (3)
and so web requests are the only basis for doing anything)
6. Define services (or "resources" or whatever -- the name "resource"
doesn't make as much sense to me, but that's bike shedding).  These are
things the app can't provide for itself, but requires (or perhaps only
wants; e.g., an app might be able to use SQLite, but could also use
PostgreSQL).  While the list of services will increase over time, without a
basic list most apps can't run at all.  We also need a core set as a kind of
reference implementation of what a fully-specified service *is*.
7. In Silver Lining I've distinguished active services (like a running
database) from passive resources (like an installed binary library).  I
don't see a reason to conflate these, as they are so very different.  Maybe
this is part of why "resource" strikes me as an odd name for something like
a database.


So... there's kind of some thoughts about process.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/web-sig/attachments/20110413/8bdac94a/attachment.html>