Automated detection of structure is provided in ultrasound M-mode imaging. A coarse and fine search for structure is used. For example, a less noise susceptible initial position or range of positions for a given structure is determined. This position is then refined. The coarse positioning and/or the refined position may use machine-trained classifiers. The positions of other structure may be used in either coarse or fine positioning, such as using a Markov Random Field. The structure or structures may be identified in the M-mode image without user input of a location in the M-mode image or along the line.