A headphone system includes a calibration microphone for performing a calibration routine with a user. The calibration microphone receives a stimulus signal emitted by the headphone system and generates a response signal indicating variations in the stimulus signal that arise due to physiological attributes of the user. Based on the stimulus signal and the response signal, the calibration engine generates response data. The calibration engine processes the response data based on a headphone transfer function (HPTF) associated with the headphone system in order to create an inverse filter that can reduce or remove acoustic variations caused by the headphone system. The calibration engine generates a personalized HRTF for the user based on the response data and the inverse filter. The personalized HRTF can be used to implement highly accurate 3D audio and is thereby well-suited for applications to immersive audio and audio-visual entertainment.