Hermetic environments

[DL Neil]

On 25/07/19 4:43 PM, Cameron Simpson wrote:
> On 25Jul2019 15:45, DL Neil <PythonList at DancesWithMice.info> wrote:
>>> In my current life I'm working on a project with a python API and a 
>>> JavaScript front end. A release involves building a clean versioned 
>>> directory on the server machine; it contains a specific Python venv 
>>> inside it; the upper layer is the encapsulation. Example:
>>>
>>>  STAGING -> app/version2
>>>  app-version
>>>    venv/....
>>>    webapp/javascript-here...
>>>    ...
>>>  app-version2
>>>    venv/....
>>>    webapp/javascript-here...
>>>    ...
>>>
>>> I still want the venv because it encapsulates the Python arena's 
>>> state of play.
>>
>>
>> Do you use a VCS, eg git or Subversion? Thus, have you disciplined 
>> yourself to check-in work, and subsequently NOT to work on your (old) 
>> copy, but to check-out a fresh copy?
> 
> Well of course. Mercurial or git. That's what makes the deploy process 
> fairly easy, one deploys from a revision with a release tag.
> 
>> Similarly, rather than adding a second environment or updates to an 
>> existing (prod) VM, it is a 'discipline' to make a copy of the 
>> appropriate VM and work with the (new) copy! (either upgrading some 
>> component of the source, the Python eco-system, or the OpSys)
>>
>> Copying/backing-up a VM is a rapid operation. So, why would you prefer 
>> to set up a second and separate py-env within an existing environment?
> 
> 1: it is smaller and much lower overhead. How many _live_ VMs are you 
> keeping around?


A pyenv is significantly smaller, but if we're including non-Python 
components in some system, 'size' increases accordingly. However, the 
'saving' in either copy-time or storage-space is not significant.

The important point here, is that the amount of time-taken in copying 
the env, might be considerably less than installing and verifying a new 
version of Python, pip-ing, upgrading non-Py components; and thereafter 
the copy and upgrade tasks are likely insignificant within the next 
sprint's-worth of dev-effort!

A VM provides the "hermetic" insulation under discussion, with no 
more/less effort than any/either of the v-env-s. (and includes the same 
advantages for the wider environment of the application)


> 2: no VMs in play at this end.

Dozens - haven't counted. (until you "made you look!"...) Every 
production system, for every client (don't currently have any new 
clients who are only in 'dev').

NB Am assuming by "keeping around" you are asking about "prod" versions, 
ie actually running systems.

However, as every 'prod' is backed by (usually) two others: 'dev' and 
(acceptance/user) 'test' (some of the latter are also installed in 
client's premises or on their networks/cloud). Then, there are some 
extra 'dev's which probably feature a PoC experiment or the like, 
completed, but yet to make it out of the client's "backlog".

OTOH if the client has not 'returned' for some time, it's not as if 
their VMs are sitting 'spooled-up' on one of my machines!


A quick 'find' shows quantities > three-digits. However, there are 
likely duplicates because of an active task to migrate between 
disks/archiving old projects 'paper work' - a task which I am 
assiduously finishing... Yeah right!


Now you've worried me. Am I keeping track? Should I be putting VMs 
(which in-turn likely contain a git client and tree), into some central 
git tree? Do I need to take a tea-break???


NB as mentioned earlier, VMs include dates and versionNRs in their 
name/label (cf DNS, etc) and such appears on docs, even invoices.

Oh, so that reminds me that I have an additional VM for (most of) my 
clients - which holds business docs, planning docs, maybe a SCRUM/Kanban 
board, mind-maps, etc, etc.

Also, if I'm supervising and in-house or external dev-team, we might 
have 'minor-version' VMs (which we refer to as "snapshots", even though 
this means something else in some VM-speak) of 'where-we-are-up-to' 
part-projects (the equivalent of me taking a whole branch from vcs for 
code-review purposes) - these don't tend to be 'kept' long.


NB I don't work exclusively in Python, eg DB and web work, so my 
definition of "VM" is a lot wider than Python venv-s! Also, I'm not a 
full-time programmer.

In the Python world, back-when, while starting to embrace Py3 I became 
nervous of the idea that I was developing in a separate Python version 
to that which ran aspects of my (Fedora-Linux) machines. VirtualBox 
offered an escape from such; and one VM led to another... As the saying 
goes: what were once habits, became vices!


BTW this is still a question: why the (venv) Python-only way cf 'other 
ideas for containerisation'; and is not intended as an exercise in 
self-justification or one-upmanship!
-- 
Regards =dn