Announcing OpenCV for Processing

10 July, 2013

I’m proud to announce the release of OpenCV for Processing, a computer vision library for Processing based on OpenCV.

You can download it here.

The goal of the library is to make it incredibly easy to get started with computer vision, to make it easy to experiment with the most common computer vision tools, and to make the full power of OpenCV’s API available to more advanced users. OpenCV for Processing is based on the official OpenCV Java bindings. Therefore, in addition to a suite of friendly functions for all the basics, you can also do anything that OpenCV can do.

The library ships with 20+ examples demonstrating its use for everything from face detection:

Screen Shot 2013-04-08 at 1.22.18 PM

(Code here: FaceDetection.pde)

to contour finding:

contours with polygon approximations

(Code here: FindContours.pde)

to background subtraction:

Background Subtraction

(Code here: BackgroundSubtraction.pde)

to depth from stereo:

Screen Shot 2013-04-12 at 2.27.30 AM

(Code here: DepthFromStereo.pde)

and AR marker detection:

Screen Shot 2013-04-12 at 12.20.17 AM

(Code here: MarkerDetection.pde)

So far, OpenCV for Processing has been tested on 64-bit Mac OSX (Lion and Mountain Lion, specifically) as well as 32-bit Windows 7 and 64- and 32-bit Linux (thanks Arturo Castro). Android support hopefully coming soon (pull requests welcome).

It’s already been used in the software for Kinograph, Matt Epler’s DIY film scanner.

OpenCV for Processing was made possible by the generous support of the Processing Foundation and O’Reilly Media. I’ve received invaluable guidance along the way from Dan Shiffman, Andres Colubri, Kyle McDonald, and Golan Levin. A lot of the library’s style was inspired by Kyle McDonald’s excellent ofxCv.

A Book!

While the documentation for OpenCV for Processing may look slim at the moment, I’m working on remedying that in a big way. I’m currently under contract with O’Reilly to write an introduction to computer vision, which will act as comprehensive documentation for OpenCV for Processing as well as a general introduction to the field of computer vision.

I’ve already begun work on the book and I’m really excited about it. It will be available through Atlas, O’Reilly’s new online learning environment. As befits a book about computer vision, it’ll make extensive use of multimedia and interaction. I’m also proud to announce that I’ve worked with O’Reilly to ensure that the book will be Creative Commons licensed from its inception. It will live on Github and accept contributions and corrections from the community. Watch this repo for details.

Why a New OpenCV Library for Processing?

Previously, there have been two OpenCV libraries for Processing, both of them French in origin.

There’s the venerable library by Atelier Hypermedia. This library was based on OpenCV 1.0 and hasn’t been updated in quite awhile. It never made the jump to Processing 2.0.

There’s also JavacvPro, which is based on JavaCV, a widely used Java wrapper for OpenCV. While I’ve used JavacvPro successfully in projects before, it has a number of shortcomings. It requires its user to build OpenCV from source, which is a major stumbling block, especially for the typical Processing user. OpenCV for Processing, on the other hand, bundles OpenCV so it installs like any other Processing library. While JavacvPro uses a relatively recent version of OpenCV, it is written in an older style, using OpenCV classes that require manual memory management. The result is that JavacvPro leaks memory and has some other erratic runtime behaviors. OpenCV for Processing uses the official Java API, which only provides access to modern memory-managed OpenCV structures. Hence, it benefits from the memory correctness and efficiency of the OpenCV developers (who are much smarter than I could ever hope to be) and doesn’t have (known) memory leaks.

Finally, JavacvPro depends on JavaCV, which slows the rate at which it keeps up with changes in the OpenCV API, and also makes it impossible for end-users to benefit from the huge amount of OpenCV documentation and support available online. Users of OpenCV for Processing can simply open the official OpenCV javadocs and start calling functions.

Caveats and Concerns

UPDATE: This section describes a problem that was present when this library was released in July of 2013. As of Processing 2.0.3 (circa Fall 2013) this problem is fixed and OpenCV for Processing should work fine with any subsequent version of Processing.

OpenCV for Processing is currently at version 0.4. It most certainly has bugs and could use serious improvement. Please find these bugs and tell me about them!

The most significant known problem is, thankfully, a temporary one that most affects users on Macs with Retina displays attempting to process video.

In the official release of Processing 2.0, the Capture and Movie libraries don’t provide access to the pixels[] array in the OpenGL-based renderers (P2D and P3D). This is a temporary stop-gap condition that will be fixed in the next release of Processing (hopefully coming in the next few weeks).

On non-retina machines, you can fix the problem by switching to the JAVA2D renderer. However, that renderer doesn’t work on Retina Macs. If you’re on a Retina Mac, you have two options: you can build Processing from source or download the older 2.0b8 version.

Hopefully all of this will be fixed soon due to Andres’s amazingness and we can forget about it.

Enjoy playing with OpenCV for Processing and be sure to show me what you build!