An in vivo motion tracking device tracking an in vivo motion that is a tracking target included in an ultrasonic image includes an image acquiring unit that is configured to acquire an ultrasonic image, an advance learning unit that is configured to perform advance learning using the ultrasonic image as learning data, and a tracking unit that is configured to track a position of the tracking target in an ultrasonic image including the tracking target after the advance learning performed by the advance learning unit.