'flattening' functions for remote execution

[holger krekel]

Brian Sturk wrote:
> I've been tinkering lately with sending python bytecode
> over a socket and executing it which works quite well.
> I'm using marshal etc to accomplish this.

I have done stuff like this too and it really is fun.
 
> What'd I'd like to be able to do is something sort of analagous
> to inlining in C.  It seems that the func_code of a function
> only has names when referencing other functions etc which
> makes sense.  What I'd like to do is something like this:
> 
> Have a function like foo below (very simple example)
> 
> import bar
> 
> def foo():
>     print 'in foo'
> 
>     bar.hello()
> 
> and send it to another python interpreter with the logic
> to handle executing this, most likely over a network and
> have the bar.hello() function resolve without
> having to have the bar module on the other end, or have the
> bar.hello function inlined.  

I have come across this issue, too. 

I think that we really need the module bar and everything it depends
on, on the 'unpickle'-side. 

For example Irmen de Jong's "Pyro" catches an ImportError on the 
unpickle-side and requests the transmission of the missing module 
and so on until it has all neccessary modules. This works ok but 
is not elegant and not very powerful. 

In reality I always want finer grained control of the modules/code
that are used on the unpickle-side.

The main problem is to get a 'transitive closure' for all the
code a function uses. (that's a tad more scientific for 'flattening 
functions' or 'inlining' bytecode :-)

I haven't found an easy solution and, alas, not even a complex one.  
My next *try* would be to parse the code and find 'import'-statements 
whose contents i'd to add to the pickle-package.  

Sometimes i have the feeling that a good solution involves 
extending/rewriting the import machinery.  

If anyone has any other ideas, let me/us know,

> etc and don't think they'll accomplish this from what I've read
> so far.  Is there any way to do this?  I'm no expert, and I've 
> run out of ideas...  I've been looking at xml-rpc too but just 
> wanted to see if I could do it with straight python and sockets.

no need for xml-rpc here, anyway.  my server-code consisted of
15 lines of code and my client code was 40 lines [*].  Up until
to the 'import' problem it is really very straightforward.

have fun,

    holger

[*] later on this month we might share some code if you are
    interested.