A method and system for analyzing video data in a security system. An analysis compares a current frame to a background model. The analysis system compares the background model to the current frame to identify changed pixel patches. The analysis system uses morphological image processing to generate masks based on the changed pixel patches. Next, the analysis system applies the masks to the background model and the current frames to determine whether the changed pixel patches are characteristic of abandoned or removed objects within the video data.