Running autogenerated code in another python instance

[Paul Cochrane]

On Thu, 03 Nov 2005 19:56:48 +0000, Tom Anderson wrote:

> On Thu, 3 Nov 2005, Paul Cochrane wrote:
> 
>> On Wed, 02 Nov 2005 06:33:28 +0000, Bengt Richter wrote:
>>
>>> On Wed, 2 Nov 2005 06:08:22 +0000 (UTC), Paul Cochrane <cochrane at shake56.esscc.uq.edu.au> wrote:
>>>
>>>> I've got an application that I'm writing that autogenerates python 
>>>> code which I then execute with exec().  I know that this is not the 
>>>> best way to run things, and I'm not 100% sure as to what I really 
>>>> should do.  I've had a look through Programming Python and the Python 
>>>> Cookbook, which have given me ideas, but nothing has gelled yet, so I 
>>>> thought I'd put the question to the community.  But first, let me be a 
>>>> little more detailed in what I want to do:

> Paul, this is a rather interesting problem. There are two aspects to it, 
> which i believe are probably separable: getting instructions from the 
> client to the server, and getting data back from the server to the client. 
> The former is more complex, i think, and what's attracted the attention so 
> far.
Tom, thanks heaps for your reply!

> The first thing i'd say is that, while eval/exec is definitely a code 
> smell, that doesn't mean it's never the right solution. If you need to be 
> able to express complex things, python code might well be the best way to 
> do it, and the best way to evaluate python code is eval/exec.
After looking at the problem a bit more I've come up with a simpler
solution to the problem I initially posted; and it involves exec.  I've
realised that what I've been doing wrong is to compile the code first
before I exec it.  If I just exec the generated code (within a predefined
namespace) then pyvisi does all of the things I want it to do without
needing separate processes for generating and rendering the code.  This
has also solved my data passing problem.  I now just need to pass a
reference to the data into the namespace where I'm running the
generated code and it all works really nicely.  Basically the problem had
been staring me in the face for ages and I just hadn't seen it.  (duh!)

>>> It's a little hard to tell without knowing more about your user input 
>>> (command language?) syntax that is translated to or feeds the process 
>>> that "autogenerates python code".
>>
>> It's basically just a command language I guess.
> 
> Hang on - the stuff that the user writes is what you're calling "pyvisi 
> code", is that right? That doesn't look like 'just a command language', 
> that looks like python, using a library you've written. Or is there 
> another language, the "just a command language", on top of that?
You're right, it isn't a command language (I looked up the cmd module in
Python in a Nutshell and then realised that a large part of my reply was
in the "I guess" part).  It is just python code that is being run.
>
> And what you call "vtk-python code" - this is python again, but using
> the renderer's native library, right?
That's correct.
> 
> And you generate the vtk-python from the pyvisi-python by executing the
> pyvisi-python, there being (pluggable renderer-specific) logic in the
> guts of your pyvisi classes to emit the vtk-python code, right? You're
> not parsing anything?
No parsing going on.  Just a translation of high-level ideas into the low
level of the renderer underneath.
> 
>>> There are lots of easy things you could do without generating and
>>> exec-ing python code per se.
>>
>> I'd love to know of other options.  I like the idea of generating the
>> code one would have to write for a particular renderer so that if the
>> user wanted to, they could use the autogenerated code to form the basis
>> of a specific visualisation script they could then hack themselves.
> 
> If you want vtk-python code as an intermediate, i think you're stuck
> with eval/exec [1].
So do I.  And as I said above, it seems to have simplified my problem
significantly.  I've also managed to speed the code up as well!

Thanks heaps for the rest of your comments and suggestions.  They've been
really helpful for me to see alternative and probably cleaner ways of
doing what I want to do.

> To be honest, i'd go with exec.
:-)  

Again, thanks for your help, I really do appreciate it.

Paul