I've polished up the mean-shift tracker code and checked it in to deface. This allows real-time histogram based tracking of arbitrary sized regions in an updating bitmap. As this is part of deface, it is under the MIT license (as opposed to GPLv2 as the footer would otherwise imply).

Here's a video of me demonstrating 2 independent trackers running on the same webcam video.

The mean-shift algorithm is one of the classic (if not THE classic) object tracking algorithms. The basics of it go like this: Define a model image (the region you'd like to track). Generate a histogram of this image, weighted according to some function of pixel coordinates. On each frame, get the mean of the pixel coordinates in the tracked region, weighted by the histogram. Repeat until mean position is "near-enough" the center of the region.

This is, in practice, very fast.

There are some nuances, such as reducing the histogram space from 256x256x256 to 16x16x16. This improves not only performance, but allows a crucial amount of "fuzziness" to the pixel value matches, which provides robustness to lighting conditions and general imprecision.

There are features I have not yet implemented, such as using the bhattarcharyya coefficient as a measure of histogram similarity to improve convergence. I've also been reading up about techniques to make the tracker also track changes in scale, though I'm not exactly sure how I'm going to ultimately approach that.

Here, play with the demo. Click and drag to create a rectangular tracking area. Release to begin tracking whatever's in that area when you release it. It works best if you make the areas just INSIDE the tracking objects so as to exclude erroneous background pixels.

I've been able to track 4 simultaneous areas in one session without noticeable slowdown. I could only find 4 things to track, so who knows how many it can comfortably do. It DOES get slower if it loses an object, because the tracker must complete all max-iterations iterations. In this case, it throws a TrackerEvent.LOST event, so you can manage that (perhaps re-initialize the tracking area with a detector).

Grab the source. Or check out deface