[execnet-dev] [discuss] Tersely reimagining execnet gateways for persistent services

[Ronn Pfannschmidt]

On 04/25/2015 11:35 AM, holger krekel wrote:
> Hi Ronny,
>
> On Fri, Apr 24, 2015 at 20:17 +0200, Ronn Pfannschmidt wrote:
>> Hello everybody,
>>
>> a while back i had an idea for getting execnet into shape for running
>> persistent services
>>
>> for that decided to imagine the gateway itself in a new shape
>>
>> currently a gatway is something we have that has
>>
>> * communication channels
>> * code execution
>>
>> now, what if a gatway itself has *only* communication channels
>> so in order to have code execution one would have to
>> install a service to the gateway,
> Here you are beginning to loose me.  The basic gateway.remote_exec(somecode)
> call is kind of an installation of a service.  How do you want to break
> it down further?
for something that manages virtualenvs, files and an starts other gateways,
it seems like a good idea, if remote_exec was not actually possible -
simply to make it more reliablee

my definition of service is basically something the remote side that
listens to command and acts on them,
remote_exec by that definition is already a service that provides
executing code remotely

- for the manager processes of a more persistent network hover code
exection is something one would like to avoid


>
>> now even installation of services into a gatway could be a service
>>
>> so details like
>>
>> * servce installation
>> * remote code execution
>> * file delivery
>> * virtualenv management
>> * gateway io startup
>> could all be services, one could pick and choose for somemthing like a
>> persistent process
> What exactly do you mean with "persistent" process?
>
> holger
>
>> suddenly new kinds of setups are thinkable
>>
>> for example
>>
>> * a ssl socket serving gateway that can only manage virtualenvs, open
>> sub io's,
>>   and tell the supervisord that manages its connect socket to restart it
>> or everything
>>
>> * having a predefined ssh command for users that only provides a service
>>   for socket connections to certain inner hosts without exposing ssh
>> functionality or other execnet features
>>
>> * having a deployent process service that synchronizes
>>   a set of wheels to the server farm and manages virtualenvs
>>   for the last N deployed versions of an application and their migration
>> state,
>>   however only providing logs and metadata to the outside users,
>>   while not allowing any commands or further gateways
>>
>>
>> in order to facilitate this change, execnet needs to grow a few extra
>> cappabilities
>>
>> as a first step, io and services on top of the io need to be more
>> cleanly separated
>>
>> * code execition needs to be a service
>> * rsync should b a service
>> * gateway via opening needs to be a service
>>
>> the nice detail about those changes is, that even though they would
>> require a larger internal change, the external facade would look the same
>>
>> once the base is created, we could then experiment with creating of
>> gateways,
>> delivery of services to gateways, as well as meta information about
>> services.
>>
>> i believe tool like batou could also be interested in similar setups
>>
>>
>> with this mail i hope i can spark some ideas, provoke thoughts and
>> receive some start initial critics
>>
>> -- Ronny
>>
>> _______________________________________________
>> execnet-dev mailing list
>> execnet-dev at python.org
>> https://mail.python.org/mailman/listinfo/execnet-dev
>>