Embodiments are generally directed to neural network training for library-based critical dimension metrology. An embodiment of a method includes optimizing a threshold for a principal component analysis of a spectrum data set to provide a principal component value, estimating a training target for one or more neural networks, training the one or more neural networks based both on the training target and on the principal component value provided from optimizing the threshold for the principal component analysis, and providing a spectral library based on the one or more trained neural networks.