A deposit detection device according to an embodiment includes a calculation module, an extraction module, and a decision module. The calculation module calculates a brightness average for each of small regions into which a predetermined region of an image captured by an imaging device is divided. The extraction module extracts, as a high-brightness region, the small region in which the brightness average calculated by the calculation module is equal to or larger than a predetermined value. The decision module decides on a brightness threshold value for detecting a deposit region corresponding to a deposit adhering to the imaging device, based on the area of the high-brightness region extracted by the extraction module.