A method for estimating a boundary between a background and a user object in a video, in which the user object is segmented from the background is provided. The initial object contour is estimated in a first video frame for object segmentation, and color probability density functions are estimated for the user object and the background segmented from the first video frame. An edge strip with a predetermined width is generated in a current frame based on an object contour of a previous frame, and a coherence probability density function of the edge strip is estimated. The user object is segmented in the current frame by energy minimization.