A signal detection apparatus and method in a Multiple Input Multiple Output (MIMO) wireless communication system are provided. The apparatus includes a receiver for receiving training symbol vectors and a data signal vector from a transmitting end through a plurality of receive (Rx) antennas and a detector for expanding the training symbol vectors into second- or higher-order polynomials, for generating a weighting parameter vector for multivariate polynomial expansion of the data signal vector by using the polynomials expanded from the training symbol vectors and thereafter for expanding the data signal vector into a second- or higher-order multivariate polynomial using the weighting parameter vector, and for detecting Transmit (Tx) values of the data signal vector from the multivariate polynomial. Therefore, signal detection performance in a Multiple Input Multiple Output (MIMO) wireless communication system resembles that of a non-linear scheme while complexity resembles that of a linear scheme.