Running autogenerated code in another python instance
Paul Cochrane
cochrane at esscc.uq.edu.au
Wed Nov 2 23:23:53 EST 2005
On Wed, 02 Nov 2005 06:33:28 +0000, Bengt Richter wrote:
> On Wed, 2 Nov 2005 06:08:22 +0000 (UTC), Paul Cochrane <cochrane at shake56.esscc.uq.edu.au> wrote:
>
>>Hi all,
>>
>>I've got an application that I'm writing that autogenerates python code
>>which I then execute with exec(). I know that this is not the best way to
>>run things, and I'm not 100% sure as to what I really should do. I've had a
>>look through Programming Python and the Python Cookbook, which have given me
>>ideas, but nothing has gelled yet, so I thought I'd put the question to the
>>community. But first, let me be a little more detailed in what I want to
>>do:
>>
Bengt,
Thanks for your reply!
> It's a little hard to tell without knowing more about your
> user input (command language?) syntax that is translated to
> or feeds the process that "autogenerates python code".
Ok, I'll try and clarify things as much as I can.
> E.g., is it a limited python subset that you are accepting as input,
> or a command language that you might implement using the cmd module, or ?
It's basically just a command language I guess. Perhaps it's best to
explain using an example. Here's the pyvisi code that generates a very
simple line plot using the vtk renderer module:
"""
Example of plotting lines with pyvisi
"""
# set up some data to plot
from Numeric import *
x = arange(10, typecode=Float)
y = x**2
# example code for how a user would write a script in pyvisi
from pyvisi import * # base level visualisation stuff
# import the objects to render the scene using the specific renderer
#from pyvisi.renderers.gnuplot import * # gnuplot
from pyvisi.renderers.vtk import * # vtk
#from pyvisi.renderers.plplot import * # plplot
# define the scene object
# a Scene is a container for all of the kinds of things you want to put
# into your plot for instance, images, meshes, arrow/vector/quiver plots,
# contour plots, spheres etc.
scene = Scene()
# create a LinePlot object
plot = LinePlot(scene)
# add some helpful info to the plot
plot.title = 'Example 2D line plot'
plot.xlabel = 'x'
plot.ylabel = 'x^2'
plot.linestyle = 'lines'
# assign some data to the plot
plot.setData(x, y)
# render the scene to screen
scene.render(pause=True, interactive=True)
# save the scene out to file
## png
plot.setData(x, y) # have to do this now because we've already
# render()ed the scene, will be removed in the
# future
scene.save(fname="simpleLinePlot.png", format=PngImage())
This code then gets translated by the pyvisi module into the vtk-python
code:
import vtk
from Numeric import *
# LinePlot.__init__()
_plot = vtk.vtkXYPlotActor()
_renderer = vtk.vtkRenderer()
_renderWindow = vtk.vtkRenderWindow()
_renderWindow.AddRenderer(_renderer)
_renderWindow.SetSize(640,480)
_renderer.SetBackground(1,1,1)
# Renderer._initRendererModule
# LinePlot.setData()
_x = array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0])
_xData = vtk.vtkDataArray.CreateDataArray(vtk.VTK_FLOAT)
_xData.SetNumberOfTuples(len(_x))
_y0 = array([0.0, 1.0, 4.0, 9.0, 16.0, 25.0, 36.0, 49.0, 64.0, 81.0])
_y0Data = vtk.vtkDataArray.CreateDataArray(vtk.VTK_FLOAT)
_y0Data.SetNumberOfTuples(len(_y0))
for i in range(len(_x)):
_xData.SetTuple1(i,_x[i])
for i in range(len(_x)):
_y0Data.SetTuple1(i,_y0[i])
_fieldData0 = vtk.vtkFieldData()
_fieldData0.AllocateArrays(2)
_fieldData0.AddArray(_xData)
_fieldData0.AddArray(_y0Data)
_dataObject0 = vtk.vtkDataObject()
_dataObject0.SetFieldData(_fieldData0)
_plot.AddDataObjectInput(_dataObject0)
_plot.SetXValuesToValue()
_plot.SetDataObjectXComponent(0,0)
_plot.SetDataObjectYComponent(0,1)
_plot.GetXAxisActor2D().GetProperty().SetColor(0, 0, 0)
_plot.GetYAxisActor2D().GetProperty().SetColor(0, 0, 0)
_renderer.SetBackground(1.0, 1.0, 1.0)
_lut = vtk.vtkLookupTable()
_lut.Build()
_colours = []
_colours.append(_lut.GetColor(0))
_plot.SetPlotColor(0, _colours[0][0], _colours[0][1], _colours[0][2])
_plot.SetPosition(0.1, 0.1)
_plot.SetWidth(0.8)
_plot.SetHeight(0.8)
# Scene.render()
_lut = vtk.vtkLookupTable()
_refLut = vtk.vtkLookupTable()
_lut.Build()
_refLut.Build()
for _i in range(256):
_lut.SetTableValue(_i, _refLut.GetTableValue(255-_i))
_iRenderer = vtk.vtkRenderWindowInteractor()
_iRenderer.SetRenderWindow(_renderWindow)
# LinePlot.render()
_renderer.AddActor2D(_plot)
_plot.SetTitle('Example 2D line plot')
_plot.SetXTitle('x')
_plot.SetYTitle('x^2')
_renderWindow.Render()
_iRenderer.Start()
Which is pretty ugly for the user to have to get to grips with, so I'm
trying to simplify the interface to the back end. I'm writing other
backends so that one only needs to change one line of code to then use
gnuplot or plplot as the renderer to generate the plot. Of course some
renderers can do things others can't, but that's another issue...
> There are lots of easy things you could do without generating and exec-ing
> python code per se.
I'd love to know of other options. I like the idea of generating the code
one would have to write for a particular renderer so that if the user wanted
to, they could use the autogenerated code to form the basis of a specific
visualisation script they could then hack themselves. I also like it
because I can see how the pyvisi code maps directly to the back end.
> How complex is a user session state?
Not 100% sure. My intention is that this is just a simple matter of
translating the commands specified at the high level into the lower level
stuff in the background. It can handle plotting several different graphs in
a loop as the data used to generate the graph changes, and it can handle
interactive display of the graphics in vtk (a vtk feature really, via
OpenGL), but nothing too tricky there. exec() is handy for running
commands generated on the fly, however, it doesn't keep the state of the
interpreter and so any variables generated on a previous call are lost.
Also, exec ignores globals etc (at least AFAIK). What would be great
would be to run the generated commands within the same scope and state,
then I wouldn't need to worry about this separate thread/process idea. I
just don't know how to do that.
> What modifies it?
> What actions are possible? History? Undo?
There's no history or undo, just processing of the script.
> Is the data a static passive
> resource to view, or partly generated or made accessible by prior actions
> in a session?
The data can be generated by prior actions in a given python script, but
basically pyvisi is set up to grab the data it's passed, and then work out
how best to render it. So in some sense the data is static and passive,
however can change at some later date, but one has to recall the setData()
method and then the render() method of the scene object to process any
changes that have occured in the data.
> Single user or collaborative?
Definitely single user.
> Shared access to everything or
> only to static data? Etc., etc.
Preferably only shared access to static data, but I do some variable name
mangling to try and keep autogenerated variables separate to anything the
user has created themselves.
> Some examples of user input and corresponding generated python might help,
> with some idea of the kind of visualization aspects being controlled ;-)
With my initial post I didn't want to put in examples as it was already
quite long and I didn't want to put people off replying. btw: I really
appreciate your feedback; I usually work best bouncing ideas off people to
find a clear way to do something.
One of the main ideas of the module is to distill the common visualisation
tasks down to a simple set of commands, and then let the interface work out
how to actually implement that. One wants to be able to put titles, and
labels and axes on things, one wants to plot 3D surfaces of data, arrows of
vector fields, ellipses of tensor data, isosurfaces of 3D scalar data, and
be able to display this to the screen and save to file. Other visualisation
aspects to be controlled are such things as camera angles, number of
contours in a plot, different kinds of colourmaps, etc.
Just to try and be of some help, here's another example of plotting some
data and its output. This time it's plotting of a random vector field with
arrows in three dimensions. First the pyvisi code:
"""
Example of plotting a 3D vector field with pyvisi
"""
# set up some data to plot
from Numeric import *
dim = 10
# initialise the positions of the vectors
x = zeros((dim,dim), typecode=Float)
y = zeros((dim,dim), typecode=Float)
z = zeros((dim,dim), typecode=Float)
# initialise the vector displacements
# (I may need to rethink how this works in the interface)
dx = zeros((dim,dim), typecode=Float)
dy = zeros((dim,dim), typecode=Float)
dz = zeros((dim,dim), typecode=Float)
# set the positions randomly, and set the displacements to some smaller
# random number but of mean zero instead of distributed between 0 and 1
import random
random.seed()
for i in range(dim):
for j in range(dim):
x[i,j] = random.random()
y[i,j] = random.random()
z[i,j] = random.random()
dx[i,j] = (random.random()-0.5)/5.0
dy[i,j] = (random.random()-0.5)/5.0
dz[i,j] = (random.random()-0.5)/5.0
# example code for how a user would write a script in pyvisi
from pyvisi import * # base level visualisation stuff
# import the objects to render the scene using the specific renderer
#from pyvisi.renderers.gnuplot import * # gnuplot
from pyvisi.renderers.vtk import * # vtk
#from pyvisi.renderers.povray import * # povray
# define the scene object
# a Scene is a container for all of the kinds of things you want to put
# into your plot for instance, images, meshes, arrow/vector/quiver
# plots,
# contour plots, spheres etc.
scene = Scene()
# create a ArrowPlot3D object
plot = ArrowPlot3D(scene)
# add some helpful info to the plot
plot.title = 'Example 3D arrow/quiver/vector field plot'
plot.xlabel = 'x'
plot.ylabel = 'y'
plot.zlabel = 'z'
# assign some data to the plot
plot.setData(x, y, z, dx, dy, dz)
# render the scene to screen
scene.render(pause=True, interactive=True)
# save the scene out to file
plot.setData(x, y, z, dx, dy, dz) # have to do this because we've already
# render()ed the scene. This requirement
# will be removed in the future
scene.save(fname="arrowPlot3D.png", format=PngImage())
And now the generated vtk-python code:
import vtk
from Numeric import *
_renderer = vtk.vtkRenderer()
_renderWindow = vtk.vtkRenderWindow()
_renderWindow.AddRenderer(_renderer)
_renderWindow.SetSize(640,480)
_renderer.SetBackground(1,1,1)
# Renderer._initRendererModule
# ArrowPlot3D.__init__()
# ArrowPlot3D.setData()
_x = array([0.877228328994, 0.412795474983, 0.163560730946, 0.89780023349,
0.129244456665, 0.40180234598, 0.542135110974, 0.46894547296,
<snip>
_y = array([0.750790237727, 0.355648864325, 0.659517997105, 0.0381646378866,
<snip>
_z = array([0.26125343592, 0.941964065844, 0.510850248162, 0.131409524918,
<snip>
_dx = array([0.0474228215121, -0.0940162717243, -0.0926294230912,
<snip>
_dy = array([0.0853229942053, 0.0358464500795, -0.00461198953457,
<snip>
_dz = array([-0.0559212949124, 0.0827739088553, 0.0849306644324,
<snip>
_points = vtk.vtkPoints()
_points.SetNumberOfPoints(100)
_points.InsertPoint(0, 0.877228, 0.750790, 0.261253)
<snip>
_vectors = vtk.vtkFloatArray()
_vectors.SetNumberOfComponents(3)
_vectors.SetNumberOfTuples(100)
_vectors.SetName("vectors")
_vectors.InsertTuple3(0, 0.047423, 0.085323, -0.055921)
<snip>
_grid = vtk.vtkUnstructuredGrid()
_grid.SetPoints(_points)
_grid.GetPointData().AddArray(_vectors)
_grid.GetPointData().SetActiveVectors("vectors")
# Scene.render()
_lut = vtk.vtkLookupTable()
_refLut = vtk.vtkLookupTable()
_lut.Build()
_refLut.Build()
for _i in range(256):
_lut.SetTableValue(_i, _refLut.GetTableValue(255-_i))
_iRenderer = vtk.vtkRenderWindowInteractor()
_iRenderer.SetRenderWindow(_renderWindow)
# ArrowPlot3D.render()
_arrow = vtk.vtkArrowSource()
_maxNorm = _grid.GetPointData().GetVectors().GetMaxNorm()
_glyph = vtk.vtkGlyph3D()
_glyph.ScalingOn()
_glyph.SetScaleModeToScaleByVector()
_glyph.SetColorModeToColorByVector()
_glyph.SetScaleFactor(0.1/_maxNorm)
_glyph.SetInput(_grid)
_glyph.SetSource(_arrow.GetOutput())
_glyph.ClampingOff()
_stripper = vtk.vtkStripper()
_stripper.SetInput(_glyph.GetOutput())
_lut = vtk.vtkLookupTable()
_lut.Build()
_refLut = vtk.vtkLookupTable()
_refLut.Build()
for i in range(256):
_lut.SetTableValue(i, _refLut.GetTableValue(255-i))
_mapper = vtk.vtkPolyDataMapper()
_mapper.SetInput(_stripper.GetOutput())
_mapper.SetScalarRange(0, _maxNorm)
_actor = vtk.vtkActor()
_actor.SetMapper(_mapper)
_renderer.AddActor(_actor)
_font_size = 14
_textProp = vtk.vtkTextProperty()
_textProp.SetFontSize(_font_size)
_textProp.SetFontFamilyToArial()
_textProp.BoldOff()
_textProp.ItalicOff()
_textProp.ShadowOff()
_textProp.SetColor(0,0,0)
_titleMapper = vtk.vtkTextMapper()
_titleMapper.SetInput("Example 3D arrow/quiver/vector field plot")
_titleProp = _titleMapper.GetTextProperty()
_titleProp.ShallowCopy(_textProp)
_titleProp.SetJustificationToCentered()
_titleProp.SetVerticalJustificationToTop()
_titleProp.SetFontSize(18)
_titleActor = vtk.vtkTextActor()
_titleActor.SetMapper(_titleMapper)
_titleActor.GetPositionCoordinate().SetCoordinateSystemToNormalizedDisplay()
_titleActor.GetPositionCoordinate().SetValue(0.5, 0.95)
_renderer.AddActor(_titleActor)
_axes = vtk.vtkCubeAxesActor2D()
_axes.SetCamera(_renderer.GetActiveCamera())
_axes.SetFlyModeToOuterEdges()
_axes.SetBounds(min(_x)-_maxNorm, max(_x)+_maxNorm, min(_y)-_maxNorm,
max(_y)+_maxNorm, min(_z)-_maxNorm, max(_z)+_maxNorm)
_axes.SetXLabel("x")
_axes.SetYLabel("y")
_axes.SetZLabel("z")
_axesProp = _axes.GetProperty()
_axesProp.SetColor(0,0,0)
_axesTitleProp = _axes.GetAxisTitleTextProperty()
_axesTitleProp.ShallowCopy(_textProp)
_axesLabelProp = _axes.GetAxisLabelTextProperty()
_axesLabelProp.ShallowCopy(_textProp)
_axesLabelProp.SetFontSize(8)
_renderer.AddActor(_axes)
_renderer.ResetCamera()
_renderer.SetBackground(1,1,1)
_renderWindow.Render()
_iRenderer.Start()
As you can see from the snipping, there's good reason for me to want to
share data objects around... I know it's ugly, but it got it working
quickly (I'm a scientist, not a proper software developer...) Hopefully you
can also see that the pyvisi script is a _lot_ simpler than what is required
to generate the visualisation.
Ok. I think that's enough for one post. Again, thanks heaps for your
reply, I do appreciate it, and hopefully now you can understand my question
a bit more clearly. I look forward to hearing from you soon.
Regards,
Paul
--
Paul Cochrane
Earth Systems Science Computational Centre
University of Queensland Brisbane Queensland 4072 Australia
E: cochrane at esscc dot uq dot edu dot au
More information about the Python-list
mailing list