Methods and systems for medical diagnosis by machine learning are disclosed. Imaging data obtained from different medical techniques can be used as a training set for a machine learning method, to allow diagnosis of medical conditions in a faster a more efficient manner. A three-dimensional convolutional neural network can be employed to interpret volumetric data available from multiple scans of a patient. The imaging data can be analyzed according to a forced taxonomy and any discrepancy in the labels of the taxonomy during data analysis by machine learning and human experts can be resolved based on the forced taxonomy.