Trusted firmware on a host server is used for managing access to a hardware security module (HSM) connected to the host server. The HSM stores confidential information associated with an operating system. As part of access management, the firmware detects a boot device identifier associated with a boot device configured to boot the operating system on the host server. The firmware then receives a second boot device identifier from the HSM. The boot device identifier and the second boot device identifier are then compared by the firmware. Based on the comparison, the firmware determines that the boot device identifier matches with the second boot device identifier. Based on this determination, the firmware grants the operating system access to the HSM.