A Multiple Input Multiple Output (MIMO) wireless communication system is provided. A receiving end apparatus includes an estimator for estimating a channel between a plurality of Transmit (Tx) antennas of a transmitting end and a plurality of Receive (Rx) antennas of a receiving end; a filter for extracting a sum between interference and noise included in an Rx signal by using the estimated channel value and for computing a covariance matrix for the sum between the interference and the noise; and a detector for detecting a Tx signal by using a signal processing matrix generated from the covariance matrix.