[Chicago] How do you deploy?

[John Jacobsen]

On Feb 25, 2013, at 5:00 AM, chicago-request at python.org wrote:

> 
> Message: 6
> Date: Sun, 24 Feb 2013 23:37:25 -0600
> From: Tal Liron <tal.liron at threecrickets.com>
> To: The Chicago Python Users Group <chicago at python.org>
> 
> I've developed my own deployment container that I may present to ChiPy in
> the future.
> 
> Otherwise, I have a rule of thumb: don't use deployment solutions that
> themselves need deployment. Fabric and Puppet both break that rule, and I
> prefer not to use them.

How do you figure Fabric (which requires only a 'pip install' step on your deploy-from node, plus copying your fabric script to the same machine) does worse in this regard than (R)?ex, which presumably has its own installation and configuration step?  You have to start somewhere.  Fabric also uses SSH for its transport -- we have used it for the past few years on deployment to O(100) servers at the South Pole from one deploy server.  There are things I don't like about it (it's essentially just some convenience wrappers for scripted deployment steps, vs. the apparently more declarative syntax provided by Puppet; plus all the 'env' stuff feels clumsy to me), but I'd like to understand your point about how Fabric needs deployment whereas your (R)?ex system does not.

We use both Puppet and Fabric on our project - Puppet for the fairly stable, packaged stuff which our IT group controls, and Fabric for deploying hand-rolled stuff which changes fairly often and needs lots of fine-tuning by the developers.  The split isn't perfect but it has certainly worked well enough for us so far.

Cheers,
John

> I've used Chef in the past, and was thoroughly
> frustrated when Chef itself broke.
> 
> So, I recommend (R)ex?: http://rexify.org/
> 
> It uses SSH for its transport, so it will always work, even on clean nodes.
> And its Perl scripts are actually very friendly to use, no less so than the
> Ruby-based tools. It has great industry support, is rock-solid and has lots
> of plugins, including integration with Gearman for really big deployments
> that can benefit from concurrency (that, of course, does require deploying
> Gearman). Documentation is imperfect, but I recommend investing in learning
> it.
> 
> Also, I am assuming you want to deploy Python: you absolutely want to use
> virtualenv, and also Supervisor if you need to run multiple servers.
> 
> Finally, in terms of versioning, consider packaging in .deb or .rpm, and
> managing your own repository. More learning up front, for sure, but it
> makes installation/upgrades dead easy, and also safe (no orphaned files).
> 
> Another pro-tip: I strongly recommend the "configuration by script"
> approach. Instead of hardcoded (or generated) configuration files that you
> have to deploy to each node, have your Python (or other language) apps know
> how to configure themselves upon startup according to interrogation of
> their runtime environment. All cloud implementations have some sort of API
> (sometimes REST-based) for that. It's more work to do this, but it means
> that you never have to worry about configuration files, especially
> *different* configuration files per node. You just need to restart services
> in order to "reconfigure" them.
> 
> PaaS ends up costing you more in the long-run, and you may hit various
> walls in terms of flexibility. Consider rolling your own solution, which
> you completely own and can fine-tune.
> 
> -Tal
>