A computer-implemented method, a computer program product, and a computer system for multi-sample dropout in deep neural network training. A computer creates multiple dropout samples in a minibatch, starting from a dropout layer and ending at a loss function layer in a deep neural network. At the dropout layer in the deep neural network, the computer applies multiple random masks for respective ones of the multiple dropout samples. At a fully connected layer in the deep neural network, the computer applies a shared parameter for all of the multiple dropout samples. After the loss function layer in the deep neural network, the computer calculates a final loss value, by averaging loss values of the respective ones of the multiple dropout samples.