A system (100) for tracking the position and orientation (i.e., the pose) of a medical imaging device (210) is described. The system determines in real time the pose of an imaging medical device based on a combination of image data and sensor data obtained from an imaging medical device and tracking sensors (210) associated with the device. Pose is determined based on a calculated tracking pose change and a calculated image pose change and a determined reliability of the calculated tracking pose change and the calculated image pose change.