A surgical controlling system comprising: at least one location estimating means to real-time locate the 3D spatial position of at least one surgical tool, at least one movement detection means in communication with a movement database and with the location estimating means and a controller, which controls the position of at least one surgical tool, in communication with a movement database, a control database and the movement detection means. The movement database stores the 3D spatial position of each surgical tool at the present time and at at least one previous time; a tool has moved if its present position is different from its previous position. The control database stores rules to identify a movement of a tool as either an allowed movement or a restricted movement. Examples of rules include a maximum speed rule, a virtual zoom rule, a virtual rotation of scene rule, and position of tool rule.