Systems and methods are disclosed wherein a tracker is detected within a field-of-view of a localizer. In one example, a virtual line-of-sight boundary is generated based on the line-of-sight relationship between the tracker and localizer. Additionally or alternatively, a virtual field-of-view boundary is generated based on the field-of-view of the localizer. A virtual object is associated with a manipulator, a surgical tool coupled to the manipulator, or the tracker when coupled to manipulator and/or surgical tool. Controller(s) predictively determine whether a planned movement of the manipulator will cause a virtual collision between the virtual object and the virtual line-of-sight boundary and/or virtual field-of-view boundary. The controller(s) enable a response based on an outcome of predictively determining whether the planned movement will cause the virtual collision.