A framework for coarse orientation detection in image data. In accordance with one aspect, the framework trains a learning structure to recognize a coarse orientation of the anatomical structure of interest based on training images. The framework may then pass one or more current images through the trained learning structure to generate a coarse orientation of the anatomical structure of interest. The framework then outputs the generated coarse orientation of the anatomical structure of interest.