A system and method for detecting pulmonary embolisms in a subject's vasculature are provided. In some aspects, the method includes acquiring a set of images representing a vasculature of the subject, and analyzing the set of images to identify pulmonary embolism candidates associated with the vasculature. The method also includes generating, for identified pulmonary embolism candidates, image patches based on a vessel-aligned image representation, and applying a set of convolutional neural networks to the generated image patches to identify pulmonary embolisms. The method further includes generating a report indicating identified pulmonary embolisms.