Embodiments are described for securing access to a debug port of an FPGA (Field Programmable Gate Array) card installed within an IHS (Information Handling System). A remote access controller determines the status of the FPGA card debug port via a query to a management controller of the FPGA card. The remote access controller generates a passcode for the debug port and disables the debug port via a message to the management controller. The management controller detects a request, that includes a requestor password, for access to the debug port. The remote access controller authorizes the requestor's access to the debug port if the requestor password matches the generated passcode. The remote access controller disables the debug port upon each power cycle of the FPGA card or upon detecting removal of a device from the debug port.