Systems, methods and devices according to this invention include a plurality of defined modification rules for modifying a sequence of packets that form an attack on an intrusion detection system. These modification rules include both rules that expand the number of packets and rules that reduce the number of packets. The reducing rules can be applied to a given attack instance to identify one or more root attack instances. The expanding rules can then be applied to each root attack instance to generate a corpus of modified attack instances. The modification rules can preserve the semantics of the attack, so that any modified attack instance generated from the given attack instance remains a true attack. To test an intrusion detection system, the corpus of modified attack instances can be used to determine whether an intrusion detection system detects every modified attack instance.