Invention Grant
- Patent Title: Automatically determining poisonous attacks on neural networks
-
Application No.: US16571323Application Date: 2019-09-16
-
Publication No.: US11645515B2Publication Date: 2023-05-09
- Inventor: Nathalie Baracaldo Angel , Bryant Chen , Biplav Srivastava , Heiko H. Ludwig
- Applicant: International Business Machines Corporation
- Applicant Address: US NY Armonk
- Assignee: International Business Machines Corporation
- Current Assignee: International Business Machines Corporation
- Current Assignee Address: US NY Armonk
- Agency: Lieberman & Brandsdorfer, LLC
- Main IPC: G06G7/00
- IPC: G06G7/00 ; G06N3/08 ; G06N20/00 ; G06F18/23 ; G06F18/24 ; G06V10/762 ; G06V10/771 ; G06V10/776

Abstract:
Embodiments relate to a system, program product, and method for automatically determining which activation data points in a neural model have been poisoned to erroneously indicate association with a particular label or labels. A neural network is trained using potentially poisoned training data. Each of the training data points is classified using the network to retain the activations of the last hidden layer, and segment those activations by the label of corresponding training data. Clustering is applied to the retained activations of each segment, and a cluster assessment is conducted for each cluster associated with each label to distinguish clusters with potentially poisoned activations from clusters populated with legitimate activations. The assessment includes executing a set of analyses and integrating the results of the analyses into a determination as to whether a training data set is poisonous based on determining if resultant activation clusters are poisoned.
Public/Granted literature
- US20210081831A1 Automatically Determining Poisonous Attacks on Neural Networks Public/Granted day:2021-03-18
Information query