Input data for a layer of a convolutional neural network (CNN) is provided by receiving input data values to be processed in a layer of the CNN. Addresses in banked memory of a buffer are determined in which the received data values are to be stored based upon format data indicating a format parameter of the input data in the layer and indicating a format parameter of a filter which is to be used to process the input data in the layer. The received input data values are stored at the determined addresses in the buffer for retrieval for processing in the layer.