A write control unit selects, in a row or column direction, N storing units from N×N storing units for storing pixel data of N (N≧2) read lines of image pickup devices and writes the data in sets of N pixels thereto, and switches a selection direction for selecting the storing units each time writes of the data of N lines are completed. A read control unit selects, in a direction different from the selection direction, N storing units and starts parallel reads of the data of N lines during writes of the data of every N-th line. Each storing unit to be first selected in the writes of the data of every N-th line performs write and read operations using different terminals, and each of the remaining storing units performs write and read operations using a common terminal.