Disclosed are an apparatus for tracking a location of a hand, includes: a skin color image detector for detecting a skin color region from an image input from an image device using a predetermined skin color of a user; a face tracker for tracking a face using the detected skin color image; a motion detector for setting a ROI using location information of the tracked face, and detecting a motion image from the set ROI; a candidate region extractor for extracting a candidate region with respect to a hand of the user using the skin color image detected by the skin color image detector and the motion image detected by the motion detector; and a hand tracker for tracking a location of the hand in the extracted candidate region to find out a final location of the hand.