A medical robotic system includes an entry guide with articulatable instruments such as surgical tools and a camera extending out of its distal end. The camera instrument is manipulatable by a camera manipulator, which has a first mechanism for pivoting a focal point of the camera instrument about a pivot of the camera instrument and a second mechanism for positioning the pivot within a three-dimensional space in response to translational commands received from one or a coupled pair of input devices. The system also includes a controller which is configured to receive sensed movement of the input devices, and cause actuation of the first mechanism in response to the sensed movement if the system is in an orientational mode and cause actuation of the second mechanism in response to the sensed movement if the system is in a translational mode.