A medical robotic system and method for controlling the system are provided to precisely measure contact force during the minimally invasive surgery. In one embodiment, a medical robotic system includes an input device, a robotic arm assembly, a surgical instrument operably coupled to the robotic arm assembly, the surgical instrument including a contact detection unit configured to detect a contact of the surgical instrument with tissue of a patient, a current measurement unit configured to measure a joint current in the robotic arm assembly, a position/velocity measurement unit configured to measure a joint position and a joint velocity in the robotic arm assembly, an external force calculation unit configured to calculate size and direction of a contact force caused by the contact of the surgical instrument to the tissue based on the joint currents, positions and velocities measured when the contact is occurred and measured when the contact is not occurred.