Methods and systems for automatically analyzing clinical images using models developed using machine learning. One system includes a server having an electronic processor and an interface for communicating with at least one data source. The electronic processor is configured to receive training information from the at least one data source over the interface. The training information includes a plurality of images and graphical reporting associated with each of the plurality of images. Each graphical reporting includes a graphical marker designating a portion of one of the plurality of images and diagnostic information associated with the portion of the one of the plurality of images. The electronic processor is also configured to perform machine learning to develop a model using the training information. The electronic processor is also configured to receive an image for analysis and automatically process the image using the model to generate a diagnosis for the image.