Embodiments of systems and methods for reducing Keyboard, Video, and Mouse (KVM) downtime during firmware update or failover events are discussed. In some embodiments, a chassis may include: a plurality of Information Handling Systems (IHSs); a first Enclosure controller (EC); and a second EC coupled to the first EC, where the first and second ECs comprise program instructions stored thereon that, upon execution, cause the chassis to: establish a KVM session with a selected IHS via the first EC; in response to the first EC receiving a firmware update command, update and restart the second EC; notify the first EC, by the second EC, that the update and restart is completed; trigger by a first KVM process in the first EC, a second KVM process in the second EC; and take control, by the second EC, of the KVM session.