Disclosed herein are a method for tracking object in video in real time in consideration of both color and shape and an apparatus for the same. The apparatus may generate at least one of a correlation filter model and a color probability density model based on an input image, determine an initial position and an initial size of a target to be tracked based on a correlation filter response value calculated by applying the correlation filter model to the input image, calculate a joint probability based on the correlation filter response value and a color object probability that is calculated based on the color probability density model, and update an object model corresponding to the target to be tracked based on a final position and a final size of the target to be tracked, which are determined by applying the joint probability to the initial position and the initial size.