A semiconductor metrology system including a spectrum acquisition tool for collecting, using a first measurement protocol, baseline scatterometric spectra on first semiconductor wafer targets, and for various sources of spectral variability, variability sets of scatterometric spectra on second semiconductor wafer targets, the variability sets embodying the spectral variability, a reference metrology tool for collecting, using a second measurement protocol, parameter values of the first semiconductor wafer targets, and a training unit for training, using the collected spectra and values, a prediction model using machine learning and minimizing an associated loss function incorporating spectral variability terms, the prediction model for predicting values for production semiconductor wafer targets based on their spectra.