[Pythonmac-SIG] real-time video capturing and processing

[Zachary Pincus]

Hi Michael,

I see some faint traces on Google of something called "PySight" that  
can grab images from iSight cameras. There may be some Apple-provided  
frameworks for grabbing iSight images as well, which could be called  
via PyObjC. Failing that, I'm sure you can find some C-level webcam  
drivers for OS X. Writing python wrappers for these (using either  
Cython [an update to pyrex] or even just the Python API) shouldn't be  
a big trouble at all if you've any C experience. (I could provide some  
help/advice along these lines if you need.)

My suggestion is to learn a bit about numpy, the python numerical  
computing module, and it's friend scipy, which provides more domain- 
specific routines. Specifically, passing a chunk of memory  
(representing an image as an array of bytes) from a C library to numpy  
is extremely easy, using the C API or even Cython. And scipy provides  
good image-manipulation tools in its ndimage and signal sub-modules.  
These are all in C and fortran, so real-time should be achievable. The  
numpy and scipy lists have really good folks on them who know their  
stuff (including some imaging people), so if you have specific  
questions about image processing with those tools, definitely ask there.

Alternately, there are python wrappers for OpenCV, which provides a  
TON of basic to advanced imaging stuff. It can be a bit overwhelming,  
but it's good and optimized for speed. (OpenCV has webcam drivers, but  
I don't think they work on OS X.) IMO, the ctypes-based wrappers are  
likely better and easier to use/extend than the SWIG ones.

Visualization might be tricky, though...  you'll likely (?) want some  
way to blit the images to the screen at some point, and so you'll want  
to choose a GUI toolkit. You could use Cocoa with PyObjC, or tkinter,  
or wx, or whatever. Most of these can accept a blob of memory and draw  
it on screen as a image. I use Pyglet, which is a nice, thin, cross- 
platform wrapper for simple windowing and OpenGL. (No widget toolkit  
for pyglet, but I just use it to throw pixels onto a 2D surface as  
fast as possible.) As a bonus, I've written some code that allows you  
to run a pyglet window in the background, which can be controlled from  
the python interpreter. (Most windowing toolkits need their own main()  
routine to run, which precludes interactive control via a python  
shell, unless you embed that in the main(). Ipython has tools to do  
this for wx and tkinter, but it's a bit klugey, IMO.) Ask me if you  
want this code.

Zach Pincus

Postdoctoral Fellow, lab of Frank Slack
Molecular, Cellular and Developmental Biology
Yale University




On Jan 7, 2009, at 9:35 AM, Michael Graber wrote:

> hi all,
>
> first: theres a library called VideoCapture for Win32 environments  
> which makes it possible to access video-capture devices. is there  
> something similar for macs?
>
> second: i would like to do real-time video-processing (like bright- 
> spot detection, movement detection, ...) using images recorded by a  
> webcam. is python suited to do so? any references?
>
> i appreciate your hints to this quite crude question.
>
> best,
> michael
> _______________________________________________
> Pythonmac-SIG maillist  -  Pythonmac-SIG at python.org
> http://mail.python.org/mailman/listinfo/pythonmac-sig