[execnet-dev] [discuss] Tersely reimagining execnet gateways for persistent services

[Ronn Pfannschmidt]

Hello everybody,

a while back i had an idea for getting execnet into shape for running
persistent services

for that decided to imagine the gateway itself in a new shape

currently a gatway is something we have that has

* communication channels
* code execution

now, what if a gatway itself has *only* communication channels
so in order to have code execution one would have to
install a service to the gateway,

now even installation of services into a gatway could be a service

so details like

* servce installation
* remote code execution
* file delivery
* virtualenv management
* gateway io startup

could all be services, one could pick and choose for somemthing like a
persistent process

suddenly new kinds of setups are thinkable

for example

* a ssl socket serving gateway that can only manage virtualenvs, open
sub io's,
  and tell the supervisord that manages its connect socket to restart it
or everything

* having a predefined ssh command for users that only provides a service
  for socket connections to certain inner hosts without exposing ssh
functionality or other execnet features

* having a deployent process service that synchronizes
  a set of wheels to the server farm and manages virtualenvs
  for the last N deployed versions of an application and their migration
state,
  however only providing logs and metadata to the outside users,
  while not allowing any commands or further gateways


in order to facilitate this change, execnet needs to grow a few extra
cappabilities

as a first step, io and services on top of the io need to be more
cleanly separated

* code execition needs to be a service
* rsync should b a service
* gateway via opening needs to be a service

the nice detail about those changes is, that even though they would
require a larger internal change, the external facade would look the same

once the base is created, we could then experiment with creating of
gateways,
delivery of services to gateways, as well as meta information about
services.

i believe tool like batou could also be interested in similar setups


with this mail i hope i can spark some ideas, provoke thoughts and
receive some start initial critics

-- Ronny