programming video manipulation

[Doug Holton]

Olivia Robinson wrote:
> Hi,
> I am an artist who creates interactive video installations that are 
> controlled using a set of sensors, a microcontroller, computer and video 
> projector.  Currently I am using MAX/MSP and Jitter to program how 
> information collected from the sensors then is used to manipulate 
> pre-recorded video footage.
> 
> I have been investigating Python as an alternative to using 
> MAX/MSP/Jitter to program the collection of information from the sensors 
> and the resulting video manipulation.  I have not been able to find 
> information about manipulating video with Python, or collecting 
> information from a microcontroller.

Hi, here's my best guess at some tips for doing this, but it would be 
nowhere as easy as using Jitter I bet (but free instead of $850).

First, see this related project that used sensors to create music, and 
is programmed with Python.  Simon Burton could give you some advice.
http://www.pcworld.idg.com.au/index.php?id=716362867
http://arrowtheory.com/hypersense/

The most difficult thing to program on your own may be reading the 
sensor data from the computer.  It used to be simple using serial ports, 
but now we use USB.  There are no solutions here, just some pointers to 
related resources:
http://developers.coedit.net/UsbProgramming
http://developers.coedit.net/AudioProgramming

For the video and real-time video transformations, you would likely want 
to use OpenGL from Python (pyopengl).  It is possible to load a frame 
from a video file as a texture in OpenGL.  And you can manipulate it 
just like any other texture or surface.  Here are demos of this below. 
The code can be adapted to pyopengl, but there is also some 
Windows-specific and Mac-specific code to read the video format.
http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=35
http://developer.apple.com/samplecode/OpenGL_Movie/OpenGL_Movie.html