A building management system includes sensors configured to measure a plurality of monitored variables and fault detection and diagnosis (FDD) system configured to identify an operating state associated with the monitored variables. The FDD system includes a communications interface configured to receive samples of the monitored variables from the plurality of sensors. The FDD system includes a direction extractor configured to use locations, in a multidimensional modeling space, of a plurality of stored operating states to extract a direction from each of the stored operating states to each of the other stored operating states. The FDD system includes a fault diagnoser configured to use the extracted directions in a voting-based diagnosis to determine an operating state for each of the samples of the monitored variables.