Presented herein are techniques for detecting sensory outcome issues through an analysis of data representing the direction of incidence/arrival of a sensory input and inertial data representing movement of the recipient's head following detection of the sensory input. By correlating recipient head movement (including lack of movement) with the arrival direction of the sensory input, a sensory prosthesis system can determine whether or not the recipient acted as expected and, if not, whether a sensory outcome problem is present.