公开(公告)号：GB2585616A

公开(公告)日：2021-01-13

申请号：GB202016400

申请日：2019-04-10

Applicant: IBM

Inventor： TAESUNG LEE ,  IAN MICHAEL MOLLOY ,  WILKA CARVALHO ,  BENJAMIN JAMES EDWARDS ,  JIALONG ZHANG ,  BRYANT CHEN

IPC: G06N3/08 ,  G06F21/57 ,  G06N20/00

Abstract: Mechanisms are provided for evaluating a trained machine learning model to determine whether the machine learning model has a backdoor trigger. The mechanisms process a test dataset to generate output classifications for the test dataset, and generate, for the test dataset, gradient data indicating a degree of change of elements within the test dataset based on the output generated by processing the test dataset. The mechanisms analyze the gradient data to identify a pattern of elements within the test dataset indicative of a backdoor trigger. The mechanisms generate, in response to the analysis identifying the pattern of elements indicative of a backdoor trigger, an output indicating the existence of the backdoor trigger in the trained machine learning model.

公开(公告)号：GB2580579A

公开(公告)日：2020-07-22

申请号：GB202007480

申请日：2018-10-29

Applicant: IBM

Inventor： TAESUNG LEE ,  IAN MICHAEL MOLLOY ,  FARHAN TEJANI

IPC: G06N3/04 ,  G06F21/57 ,  G06N3/08

Abstract: Mechanisms are provided for providing a hardened neural network. The mechanisms configure the hardened neural network executing in the data processing system to introduce noise in internal feature representations of the hardened neural network. The noise introduced in the internal feature representations diverts gradient computations associated with a loss surface of the hardened neural network. The mechanisms configure the hardened neural network executing in the data processing system to implement a merge layer of nodes that combine outputs of adversarially trained output nodes of the hardened neural network with output nodes of the hardened neural network trained based on the introduced noise. The mechanisms process, by the hardened neural network, input data to generate classification labels for the input data and thereby generate augmented input data which is output to a computing system for processing to perform a computing operation.