A multiple input multiple output wireless communication system and a channel decomposition method thereof are provided. The wireless communication system includes a precoding unit, a channel estimation unit, a channel decomposition unit and a quantization unit. The precoding unit receives a plurality of transmission data streams and wireless channel information to provide a plurality of transmission symbols to a wireless channel. The channel estimation unit estimates the wireless channel to provide a channel matrix. The channel decomposition unit makes a plurality of elements on a diagonal and adjacent to a side of the diagonal as real numbered elements through an element ration and correspondingly rotates a first unit matrix and a second unit matrix to obtain channel state information. The quantization unit provides the wireless channel information according to the channel state information.