A focus adjustment device comprising: a first detector which detects a focused state by a contrast detection system; second detectors which detect a focused state by a phase difference detection system; and a control unit which controls the first detector and second detectors so as to detect the focused state by the second detectors when detecting the focused state by the first detector.