A speaker system to control directivity of a speaker unit using a plurality of microphones, and a method thereof. The method includes sensing through a plurality of channels a shock sound with an impulse pattern generated at a listening position and measuring delay values between signals of the channels, reading a predetermined listening position compensation filter coefficient in accordance with the measured delay values, and controlling directivity of the speaker unit by granting the read compensation filter coefficient on input audio signals.