An automated pruning technique is proposed for reducing the size of a convolutional neural network. A large-sized network is trained and then connections between layers are explored to remove redundant parameters. Specifically, a scaling neural subnetwork is connected to the neural network and designed to infer importance of the filters in the neural network during training of the neural network. Output from the scaling neural subnetwork can then be used to remove filters from the neural network, thereby reducing the size of the convolutional neural network.