A method for securing a KVM Matrix system by inserting a plurality of input security isolators, each of the input security isolators is placed between a host computer and matrix host adapter of the KVM matrix system to enforce security data flow policy that is applicable for the corresponding host computer. Additionally, a security filter is placed between peripheral devices and a matrix console adapter to enforce security data flow policy that is applicable for the corresponding peripheral devices.