Neural Networks such as Deep Neural Networks (DNNs) output calibrated probabilities that substantially represent frequencies of occurrences of events. A DNN propagates uncertainty information of a unit of the DNN from an input to an output of the DNN. The uncertain information measures a degree of consistency of the test data with training data used to train a DNN. The uncertainty information of all units of the DNN can be propagated. Based on the uncertainty information, the DNN outputs probability scores that reflect received input data that is substantially different from the training data.