[python-uk] Suggestions / best practices for deployment

Sat May 18 18:41:14 CEST 2013

Daniel, who are you disagreeing with?   Everyone here agrees on
automation, I think.

- Andy

On 18 May 2013 12:24, Daniel Pope <lord.mauve at gmail.com> wrote:
> I thoroughly disagree with those who are saying that for small
> installations, it's less time-consuming to do things manually. A
> deployment/provisioning system gives you reproducibility - an executable
> record of how to re-create a server configuration or re-run a deployment
> without missing anything. The number of times I've spent 20 minutes
> wondering what I've missed all add up - that is why it automation breaks
> even so rapidly.
>
> Of course, you (or your employers) might have other reasons to want
> automation:
>
> - so that your deployment/provisioning is subject to automated testing,
> version control, code review, etc
> - so that you can build replicas of production servers for development,
> testing or disaster recovery
> - so that you can smash and rebuild a compromised or faulty machine without
> wasting time
> - so that you can deploy a dozen times a day and get features or fixes into
> the hands of your users
>
>
> On 17 May 2013 15:39, M.-A. Lemburg <mal at egenix.com> wrote:
>>
>> On 16.05.2013 17:46, Andy Robinson wrote:
>> > Speaking as a relatively obsolete dinosaur, I would suggest that if
>> > you are going to discuss specific deployment practices, you start with
>> > the most fundamental ones:  SSH, the unix shell and so on.
>> >
>> > We have had issues over the years with people coming in and
>> > introducing sexy new deployment tools, but ultimately they all just
>> > run unix commands.  Anyone managing a web application in the
>> > non-microsoft world is ultimately depending on this.  Some key skills
>> > (assuming a Linux/Mac/Unix-ish environment):
>> > - know about SSH keys and logging into remote machines
>> > - know the basics of at least one command line editor (e.g. vi)
>> > - basic shell knowledge:  environment variables, testing for existence
>> > of files and directories etc
>> > - how to interact with your database from the command line, if you use
>> > one (including dump and restore)
>> > - how your web server works: starting, stopping, configuration files,
>> > where log files live and how it talks to Python
>> >
>> > Fabric may be useful if you want to control half a dozen machines from
>> > your desktop, and it might add a lot of value if you want to control a
>> > hundred of them.  But to update one server, you deploy by logging into
>> > it and then running commands or short scripts.
>> >
>> > For example, we have a 'demo site' we rebuild pretty often that uses
>> > Django, MYSQL, Celery and a few other things.  It runs on plain
>> > vanilla Ubuntu machines we build ourselves.  The sequence is...
>> >
>> > 1. Log in via SSH
>> > 2. CD to correct directory
>> > 3. activate virtual environment
>> > 4. stop any celery worker processes
>> > 5. stop web server processes (* in our setup, we leave Apache running)
>> > 6. pull latest code from mercurial - both the app, and 3-4 libraries
>> > it depends on
>> > 7. run a management command to rebuild the database
>> > 8. run a smallish in-place test suite
>> > 9. restart celery workers
>> > 10.restart web server
>> > 11. log out
>> >
>> > All of this after the login and CD can be handled by a shell script on
>> > the path of the server, so you can just run a command called something
>> > like
>> >   ./update_server
>> >
>> > More realistically, we tend to end up with a management shell script
>> > called 'server' with a bunch of commands/arguments like 'stop / start
>> > / restart / update-code-in-staging / copy-live-data-to-staging /
>> > run-health-checks / swap-live-and-staging' and so on.  SSH can execute
>> > remote commands like this just fine with the right arguments, if
>> > actually logging in is too tedious.
>> >
>> > Production sites are complex and all different.  You might want to do
>> > instantaneous swaps from live to staging (and be able to back out fast
>> > if stuff goes wrong); to switch DNS so the world is looking at another
>> > server while you update one; you might have large databases to copy or
>> > migrate that need significant time; it may or may not be acceptable to
>> > lose sessions and have downtime; and so on.
>> >
>> >
>> > It takes less time to learn the fundamentals than you will spend
>> > debugging why your fancy new deployment tool stopped working after
>> > some Python dependency upgrade somewhere.   And it is less likely that
>> > your new hires will disagree if you stick with the lowest common
>> > denominator.
>>
>> Fully agreed.
>>
>> The new devops tools are nice when it comes to managing farms
>> of VMs, where each VM is setup in more or less the same way,
>> and you want to reduce repetitive tasks, but for a typical
>> setup with just a few VMs/servers it'll take you longer to
>> write and (most importantly) test your devops scripts than
>> it would to write a few scripts that you can run via SSH on
>> the servers.
>>
>> No matter how smart you make your devops scripts, Murphy's law
>> is inevitably going to strike and humans are so much better at
>> parsing weird intermittent error messages than machines ...
>> still, after all these years :-)
>>
>> --
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Source  (#1, May 17 2013)
>> >>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>> >>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>> ________________________________________________________________________
>> 2013-05-07: Released mxODBC Zope DA 2.1.2 ...     http://egenix.com/go46
>> 2013-05-06: Released mxODBC 3.2.3 ...             http://egenix.com/go45
>>
>> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>>
>>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>            Registered at Amtsgericht Duesseldorf: HRB 46611
>>                http://www.egenix.com/company/contact/
>> _______________________________________________
>> python-uk mailing list
>> python-uk at python.org
>> http://mail.python.org/mailman/listinfo/python-uk
>
>
>
> _______________________________________________
> python-uk mailing list
> python-uk at python.org
> http://mail.python.org/mailman/listinfo/python-uk
>

-- 
Andy Robinson
Managing Director
ReportLab Europe Ltd.
Thornton House, Thornton Road, Wimbledon, London SW19 4NG, UK
Tel +44-20-8405-6420