A focus position detection device calculates, for each of shift amount calculation areas in a measurement area set on an image sensor of an imaging unit, a local shift amount of a second sub-image with respect to a first sub-image when a subject on the first sub-image and a subject on the second sub-image are most coincident with each other and a feature quantity of a probability distribution representing accuracy of the local shift amount, estimates the probability distribution based on the local shift amount and the feature quantity for each shift amount calculation area, combines the probability distributions of the shift amount calculation areas to obtain a distribution of accuracy of the local shift amount for the measurement area, and calculates a representative value representing the distance between a focus position produced by an optical system of the imaging unit and the image sensor based on the distribution.