[python-uk] Suggestions / best practices for deployment

Sat May 18 13:24:47 CEST 2013

I thoroughly disagree with those who are saying that for small
installations, it's less time-consuming to do things manually. A
deployment/provisioning system gives you reproducibility - an executable
record of how to re-create a server configuration or re-run a deployment
without missing anything. The number of times I've spent 20 minutes
wondering what I've missed all add up - that is why it automation breaks
even so rapidly.

Of course, you (or your employers) might have other reasons to want
automation:

- so that your deployment/provisioning is subject to automated testing,
version control, code review, etc
- so that you can build replicas of production servers for development,
testing or disaster recovery
- so that you can smash and rebuild a compromised or faulty machine without
wasting time
- so that you can deploy a dozen times a day and get features or fixes into
the hands of your users

On 17 May 2013 15:39, M.-A. Lemburg <mal at egenix.com> wrote:

> On 16.05.2013 17:46, Andy Robinson wrote:
> > Speaking as a relatively obsolete dinosaur, I would suggest that if
> > you are going to discuss specific deployment practices, you start with
> > the most fundamental ones:  SSH, the unix shell and so on.
> >
> > We have had issues over the years with people coming in and
> > introducing sexy new deployment tools, but ultimately they all just
> > run unix commands.  Anyone managing a web application in the
> > non-microsoft world is ultimately depending on this.  Some key skills
> > (assuming a Linux/Mac/Unix-ish environment):
> > - know about SSH keys and logging into remote machines
> > - know the basics of at least one command line editor (e.g. vi)
> > - basic shell knowledge:  environment variables, testing for existence
> > of files and directories etc
> > - how to interact with your database from the command line, if you use
> > one (including dump and restore)
> > - how your web server works: starting, stopping, configuration files,
> > where log files live and how it talks to Python
> >
> > Fabric may be useful if you want to control half a dozen machines from
> > your desktop, and it might add a lot of value if you want to control a
> > hundred of them.  But to update one server, you deploy by logging into
> > it and then running commands or short scripts.
> >
> > For example, we have a 'demo site' we rebuild pretty often that uses
> > Django, MYSQL, Celery and a few other things.  It runs on plain
> > vanilla Ubuntu machines we build ourselves.  The sequence is...
> >
> > 1. Log in via SSH
> > 2. CD to correct directory
> > 3. activate virtual environment
> > 4. stop any celery worker processes
> > 5. stop web server processes (* in our setup, we leave Apache running)
> > 6. pull latest code from mercurial - both the app, and 3-4 libraries
> > it depends on
> > 7. run a management command to rebuild the database
> > 8. run a smallish in-place test suite
> > 9. restart celery workers
> > 10.restart web server
> > 11. log out
> >
> > All of this after the login and CD can be handled by a shell script on
> > the path of the server, so you can just run a command called something
> > like
> >   ./update_server
> >
> > More realistically, we tend to end up with a management shell script
> > called 'server' with a bunch of commands/arguments like 'stop / start
> > / restart / update-code-in-staging / copy-live-data-to-staging /
> > run-health-checks / swap-live-and-staging' and so on.  SSH can execute
> > remote commands like this just fine with the right arguments, if
> > actually logging in is too tedious.
> >
> > Production sites are complex and all different.  You might want to do
> > instantaneous swaps from live to staging (and be able to back out fast
> > if stuff goes wrong); to switch DNS so the world is looking at another
> > server while you update one; you might have large databases to copy or
> > migrate that need significant time; it may or may not be acceptable to
> > lose sessions and have downtime; and so on.
> >
> >
> > It takes less time to learn the fundamentals than you will spend
> > debugging why your fancy new deployment tool stopped working after
> > some Python dependency upgrade somewhere.   And it is less likely that
> > your new hires will disagree if you stick with the lowest common
> > denominator.
>
> Fully agreed.
>
> The new devops tools are nice when it comes to managing farms
> of VMs, where each VM is setup in more or less the same way,
> and you want to reduce repetitive tasks, but for a typical
> setup with just a few VMs/servers it'll take you longer to
> write and (most importantly) test your devops scripts than
> it would to write a few scripts that you can run via SSH on
> the servers.
>
> No matter how smart you make your devops scripts, Murphy's law
> is inevitably going to strike and humans are so much better at
> parsing weird intermittent error messages than machines ...
> still, after all these years :-)
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, May 17 2013)
> >>> Python Projects, Consulting and Support ...   http://www.egenix.com/
> >>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2013-05-07: Released mxODBC Zope DA 2.1.2 ...     http://egenix.com/go46
> 2013-05-06: Released mxODBC 3.2.3 ...             http://egenix.com/go45
>
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
> _______________________________________________
> python-uk mailing list
> python-uk at python.org
> http://mail.python.org/mailman/listinfo/python-uk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-uk/attachments/20130518/bd77bdad/attachment.html>