A neural network is structured with a plurality of levels of nodes. Each level has a level-specific stabilization parameter that adjusts a learning rate, at a corresponding level, during training. The stabilization parameter has a value that varies inversely relative to a change in an objective training function during back-propagation of the error through the level.