An optical unit with a shake correction function include a movable body having an optical module and a holder surrounding the optical module, a swing support mechanism swingably supporting the movable body, a fixed body, and a shake correction drive mechanism. The optical module has a protruded part, and the holder has a position restriction part with which the protruded part is abutted. The fixed body includes a case accommodating the holder, and the case includes an outer frame part surrounding the holder and a projecting part projecting to an inner peripheral side from an end part on an image side of the outer frame part. The projecting part has a stopper part overlapping with the holder, and the protruded part and the position restriction part are located on an inner peripheral side with respect to the projecting part when viewed in the optical axis direction.