A physical security system having a plurality of sensors and a sensor report aggregator. The sensors may detect a large number of physical activities. The aggregator may cluster a large number of detected reports to a small number of sets of reports. The sets of reports may be reduced to hypotheses. From the hypotheses, the aggregator may develop hypotheses about the physical environment which the sensors are monitoring in view of a security reference model. The security reference model may include, but not be limited to, facility models, physical security models, and/or attack models. The hypotheses may have probabilities assigned to them according to their certitude of likelihood and severity of danger.