Running autogenerated code in another python instance

[Paul Cochrane]

On Mon, 07 Nov 2005 00:01:29 +0000, Bengt Richter wrote:

> On Thu, 03 Nov 2005 14:23:53 +1000, Paul Cochrane <cochrane at esscc.uq.edu.au> wrote:
> 
>>On Wed, 02 Nov 2005 06:33:28 +0000, Bengt Richter wrote:
>>
>>> On Wed, 2 Nov 2005 06:08:22 +0000 (UTC), Paul Cochrane <cochrane at shake56.esscc.uq.edu.au> wrote:
>>> 
>>>>Hi all,
>>>>
>>>>I've got an application that I'm writing that autogenerates python code
>>>>which I then execute with exec().  I know that this is not the best way to
>>>>run things, and I'm not 100% sure as to what I really should do.  I've had a
>>>>look through Programming Python and the Python Cookbook, which have given me
>>>>ideas, but nothing has gelled yet, so I thought I'd put the question to the
>>>>community.  But first, let me be a little more detailed in what I want to
>>>>do:
>>>>
>>
>>Bengt,
>>
>>Thanks for your reply!
>>
>>> It's a little hard to tell without knowing more about your
>>> user input (command language?) syntax that is translated to
>>> or feeds the process that "autogenerates python code".
>>Ok, I'll try and clarify things as much as I can.
>>
> [...snip great reply...]
Thanks :-)
> 
> I owe you another reply, but I started and I couldn't spend the time to
> do it justice. But in the meanwhile, I would suggest thinking about how
> the MVC (model-view-controller) concept might help you factor things.
> ISTM you are already part way there. See
>     http://en.wikipedia.org/wiki/MVC
Thanks for the tip.

> for some infos. To that I'd add that if you want a top level interactive visualization tool,
> you might want to look at it like a kind of IDE, where you might want a few workspace/project kind
> of top level commands to help manage what you would otherwise do by way of manually creating
> directory subtrees for various things etc. Anyway, if you want to use template code and edit
> it and then compile and run it, you could select templates and make working copies automatically
> and invoke some favorite editor on it from the top level command interpreter. Having these
> pieces consistently arranged in projects/workspaces and shared template spaces etc. would
> make it a single command to create a fresh space and not worry about colliding with something
> you did just to show someone a little demo, etc. This is not the central topic, but good useability
> is nice ;-)
I think this is probably a bit more complex than I really want.  All I
want to do is to take the pain out of scripting visualisation of large and
complex data sets.  My code will eventually run inside a batch job, or
behind a web-based GRID interface thing (still being developed by other
people in my group).  The interactive part of vtk is just a
handy side effect; allowing users to rotate objects within a window and
see what's on the other side etc is really nice, but not necessarily the
main aim. Nevertheless, I've found out what I've been doing wrong all
along: I've been compiling the code objects before exec-ing them.  By just
running exec on the generated code I can run the code generation and
rendering within the one process (since not compiling beforehand allows me
to maintain state), and I can pass the data around as the objects
themselves and not this convoluted convert to string, convert back to
number process I had going. It's also now a lot faster (which is always a
bonus)!

> That way if you just wanted to re-run something, you'd just select it and skip calling the
> editor. Or if a step was to generate data, you could either create the data source program
> by several steps or possibly just go on to define or invoke a visualization step with
> a particular renderer and/or output, knowing that the data source was already set up in a
> standard way. You could also consider borrowing unix piping/redirection concepts
> for some command syntax, for composition of standard interface actions (not to mention
> invoking the real thing in a subprocess when appropriate). Just free-associating here ;-)
These are great ideas, but probably a bit more than I can handle atm :-)

> Anyway, gotta go for now, sorry.
No worries!  I really appreciate your feedback.

Paul