A technology for tracking a target object using depth information is disclosed. An object image contain a target object can be obtained from a reference image. A target distance from a robot to the target object can also be derived or accessed. When the robot captures a new image, 2D matching can be performed to find the horizontal and vertical movement of the target object. Distance change or depth change of the target object can also be calculated from the new image. A depth change ratio can be calculated and be utilized to scale a bounding box. The robot can then move to a new location according to the horizontal, vertical and depth changes. A new object image can be extracted from the new image according to the scaled bounding box. The object tracking can be continued using a similar process when the next image is captured.