A fast Fourier transform (FFT) circuit of an audio processing device configured to perform an N-points FFT and including a memory circuit and a butterfly operation unit circuit is provided. The butterfly operation unit circuit reads two points input data from the memory circuit, performs a butterfly operation for the two points input data according to a twiddle factor to generate two points output data, and writes the two points output data into the memory circuit. The butterfly operation unit circuit includes a multiplier and a plurality of adders/subtractors. The multiplier sequentially multiplies real or imaginary coefficients of one of the two points input data by real or imaginary coefficients of the twiddle factor in multiple clock cycles. The multiplier performs a multiplication once in each clock cycle. The adders/subtractors perform addition/subtraction, such that the butterfly operation unit circuit generates the two points output data.