A method for training a neural network comprising at least one layer comprising a plurality of input nodes, a plurality of output nodes, and a plurality of connections for connecting each one of the plurality of input nodes to each one of the plurality of output nodes, is provided. The method comprises pseudo-randomly selecting a subset of the plurality of connections, each connection of the plurality of connections having associated therewith a weight parameter and a probability of being retained in the neural network, generating output data by feeding input data over the subset of connections, computing an error between the generated output data and desired output data, and, for at least one connection in the subset of connections, determining a contribution of the weight parameter to the error and updating the probability of being retained in the neural network accordingly.