Systems and methods analyze training of a first machine learning system with a second machine learning system. The first machine learning system comprises a neural network with a first inner layer node. The method includes connecting the first machine learning system to an input of the second machine learning system. The second machine learning system comprises a second objective function for analyzing an internal characteristic of the first machine learning system and which is different from a first objective function for the first machine learning system. The method further includes providing a training data item to the first machine learning system, collecting internal characteristic data from the first inner layer node of the first machine learning system associated with the internal characteristic, computing partial derivatives of the first objective function through the first machine learning system with respect to the training data item, and computing partial derivatives of the second objective function through both the second machine learning system and the first machine learning system with respect to the collected internal characteristic data.