A multiplication accumulating device and a method thereof are provided. The multiplication accumulating device includes a product generator, a plurality of registers, a product reducer, and an adder. The product generator performs a product operation on a multiplicand and a multiplier to generate a product result of 2N−1 columns. The product reducer is used to append data from a portion of the plurality of registers to the columns in the product result to generate an appending result of 2N−1 columns. The product reducer performs a reduction operation on the appending result according to a column height of each column in the appending result to obtain a reduced result. The product reducer renews the data in the plurality of registers according to the reduced result. The adder adds the data in the plurality of registers according to an accumulation signal to generate a multiplication accumulating operation result.