A hearing assistance system comprises a left ear device and a right ear device respectively configured to be worn by a wearer. One or more microphones are provided at each of the left and right ear devices. One or more positional sensors are configured to determine a three-dimensional position of the hearing assistance system in response to the wearer looking at a sound source in space. A user interface is configured to receive an input directly from the wearer. A memory is configured to store the three-dimensional position of the hearing assistance system in response to the received input. A processor is configured to adjust a directional polar pattern of the one or more microphones provided at one or both of the left and right ear devices in response to the stored three-dimensional position.