Systems and methods for generating a radiation therapy treatment plan by using ultrasound images are disclosed. The method includes obtaining a first set of image data representing a first image of an anatomical region that includes a treatment applicator inserted into the anatomical region. The method further includes receiving tracking information indicating a position of the treatment applicator as represented in the first set of image data, obtaining a second set of image data representing a second image of the anatomical region, and combining at least a portion of the first set of image data with the second set of image data. One of the first set and the second set of image data is used to identify a target tissue portion in the anatomical region. The method further includes generating, based on the combined image data, treatment plan information for treating the target tissue portion in the anatomical region.