Autofocusing is performed in response to a weighted sum of previous blur difference depth estimates after being adaptively fitted at each focus adjustment iteration. Variance is also determined across both past and present estimations providing a confidence measure on the present focus position for the given picture. In one embodiment focus adjustment are repeated until the variance is sufficiently low as to indicate confidence that a proper focus has been attained. The method increases accuracy and speed of focusing by utilizing previous depth estimates while adapting the matching data to overcome distortion, such as due to saturation, cut-off and noise.