A system is provided for determining vulnerability metrics for graph-based configuration security. During operation, the system generates a multi-layer graph for a system with a plurality of interconnected components. The system determines, based on the multi-layer subgraph, a model for a multi-step attack on the system by: calculating, based on a first set of variables and a first set of tunable parameters, a likelihood of exploiting a vulnerability in the system; and calculating, based on a second set of variables and a second set of tunable parameters, an exposure factor indicating an impact of exploiting a vulnerability on the utility of an associated component. The system determines, based on the model, a set of attack paths that can be used in the multi-step attack and recommends a configuration change in the system, thereby facilitating optimization of system security to mitigate attacks on the system while preserving system functionality.