[python-uk] Suggestions / best practices for deployment

[M.-A. Lemburg]

On 16.05.2013 17:46, Andy Robinson wrote:
> Speaking as a relatively obsolete dinosaur, I would suggest that if
> you are going to discuss specific deployment practices, you start with
> the most fundamental ones:  SSH, the unix shell and so on.
> 
> We have had issues over the years with people coming in and
> introducing sexy new deployment tools, but ultimately they all just
> run unix commands.  Anyone managing a web application in the
> non-microsoft world is ultimately depending on this.  Some key skills
> (assuming a Linux/Mac/Unix-ish environment):
> - know about SSH keys and logging into remote machines
> - know the basics of at least one command line editor (e.g. vi)
> - basic shell knowledge:  environment variables, testing for existence
> of files and directories etc
> - how to interact with your database from the command line, if you use
> one (including dump and restore)
> - how your web server works: starting, stopping, configuration files,
> where log files live and how it talks to Python
> 
> Fabric may be useful if you want to control half a dozen machines from
> your desktop, and it might add a lot of value if you want to control a
> hundred of them.  But to update one server, you deploy by logging into
> it and then running commands or short scripts.
> 
> For example, we have a 'demo site' we rebuild pretty often that uses
> Django, MYSQL, Celery and a few other things.  It runs on plain
> vanilla Ubuntu machines we build ourselves.  The sequence is...
> 
> 1. Log in via SSH
> 2. CD to correct directory
> 3. activate virtual environment
> 4. stop any celery worker processes
> 5. stop web server processes (* in our setup, we leave Apache running)
> 6. pull latest code from mercurial - both the app, and 3-4 libraries
> it depends on
> 7. run a management command to rebuild the database
> 8. run a smallish in-place test suite
> 9. restart celery workers
> 10.restart web server
> 11. log out
> 
> All of this after the login and CD can be handled by a shell script on
> the path of the server, so you can just run a command called something
> like
>   ./update_server
> 
> More realistically, we tend to end up with a management shell script
> called 'server' with a bunch of commands/arguments like 'stop / start
> / restart / update-code-in-staging / copy-live-data-to-staging /
> run-health-checks / swap-live-and-staging' and so on.  SSH can execute
> remote commands like this just fine with the right arguments, if
> actually logging in is too tedious.
> 
> Production sites are complex and all different.  You might want to do
> instantaneous swaps from live to staging (and be able to back out fast
> if stuff goes wrong); to switch DNS so the world is looking at another
> server while you update one; you might have large databases to copy or
> migrate that need significant time; it may or may not be acceptable to
> lose sessions and have downtime; and so on.
> 
> 
> It takes less time to learn the fundamentals than you will spend
> debugging why your fancy new deployment tool stopped working after
> some Python dependency upgrade somewhere.   And it is less likely that
> your new hires will disagree if you stick with the lowest common
> denominator.

Fully agreed.

The new devops tools are nice when it comes to managing farms
of VMs, where each VM is setup in more or less the same way,
and you want to reduce repetitive tasks, but for a typical
setup with just a few VMs/servers it'll take you longer to
write and (most importantly) test your devops scripts than
it would to write a few scripts that you can run via SSH on
the servers.

No matter how smart you make your devops scripts, Murphy's law
is inevitably going to strike and humans are so much better at
parsing weird intermittent error messages than machines ...
still, after all these years :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 17 2013)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2013-05-07: Released mxODBC Zope DA 2.1.2 ...     http://egenix.com/go46
2013-05-06: Released mxODBC 3.2.3 ...             http://egenix.com/go45

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/