Learning Patterns of Activity Using Real-Time Tracking

This work aims to build a visual monitoring system that passively observes moving objects at a site and automatically learns patterns of activity from those observations, with larger sites handled by multiple coordinated cameras. The paper concentrates on the motion-tracking component. Motion segmentation is performed with an adaptive background subtraction method that models each pixel as a mixture of Gaussians and updates the model using an online approximation, then evaluates which of the Gaussian distributions most likely correspond to the background process.

The result is a stable, real-time outdoor tracker that reliably handles lighting changes, repetitive motions from background clutter, and long-term scene changes. Although the tracker does not know the identity of the objects it follows, an object's identity is consistent throughout a tracking sequence; the system exploits this by accumulating joint co-occurrence statistics of representations within each sequence and using them to build a hierarchical binary-tree classification. This enables classifying both whole sequences and individual activity instances, and the adaptive Gaussian-mixture background model became a widely used foundation for video surveillance.