Disclosed herein, among other things, are apparatus and methods for annoyance perception and modeling for hearing-impaired listeners. One aspect of the present subject matter includes a method for improving noise cancellation for a wearer of a hearing assistance device having an adaptive filter. In various embodiments, the method includes calculating an annoyance measure or other perceptual measure based on a residual signal in an ear of the wearer, the wearer's hearing loss, and the wearer's preference. A spectral weighting function is estimated based on a ratio of the annoyance measure or other perceptual measure and spectral energy. The spectral weighting function is incorporated into a cost function for an update of the adaptive filter. The method includes minimizing the annoyance or other perceptual measure based cost function to achieve perceptually motivated adaptive noise cancellation, in various embodiments.