A filter processing apparatus comprises: K arithmetic operation units configured to perform predetermined arithmetic operation processing parallelly; a data holding unit configured to hold a plurality of data values; a coefficient holding unit configured to hold a plurality of filter coefficients; a first pattern holding unit configured to hold a first pattern that selects K data values from the plurality of data values; a second pattern holding unit configured to hold a second pattern that selects one filter coefficient from the plurality of filter coefficients; and a control unit configured to sequentially input the K data values to the K arithmetic operation units with reference to the first pattern and sequentially input the one common filter coefficient to the K arithmetic operation units with reference to the second pattern.