Disclosed herein are a method and apparatus for extracting feature points using hierarchical image segmentation and an image based localization method using the extracted feature points. An image is segmented using an affinity degree obtained using information observed during position estimation, new feature points are extracted from segmented areas in which registered feature points are not included, and position estimation is performed based on the new feature points. Accordingly, stable and reliable localization may be performed.