A wireless device uses a tracking system to determine locations of target devices relative to the wireless device in an environment including one or more target devices. The tracking system receives a signal transmitted by a target device in the environment and performs an angle of arrival operation to determine the angle of arrival of the received signal. The tracking system applies a filter to determine an estimation value for the target device representing a system state for the target device. The tracking system assigns the received signal and corresponding angle of arrival to a track based on the estimation value. The tracking system identifies and removes outlier signals from the track to produce a filtered set of signals. The filtered set of signals is used to compute a point-of-origin vector for the target device for use in interacting and communicating with the target device.