In accordance with some embodiments, systems, methods, and media for automatically segmenting and diagnosing prostate lesions using multi-parametric magnetic resonance imaging (mp-MRI) data are provided. In some embodiments, the system comprises is programmed to: receive mp-MRI data depicting a prostate, including T2w data and ADC data; provide the T2w data and ADC data as input to first and second input channels of a trained convolutional neural network (CNN); receive, from the trained CNN, output values from output channels indicating which pixels are likely to correspond to a particular class of prostate lesion, the channels corresponding to predicted aggressiveness in order of increasing aggressiveness, identify a prostate lesion in the data based on output values greater than a threshold; predict an aggressiveness based on which channel had values over the threshold; and present an indication that a prostate lesion of the predicted aggressiveness is likely present in the prostate.