A machine learning system that implements Sobolev Independence Criterion (SIC) for feature selection is provided. The system receives a dataset including pairings of stimuli and responses. Each stimulus includes multiple features. The system generates a correctly paired sample of stimuli and responses from the dataset by pairing stimuli and responses according to the pairings of stimuli and responses in the dataset. The system generates an alternatively paired sample of stimuli and responses from the dataset by pairing stimuli and responses differently than the pairings of stimuli and responses in the dataset. The system determines a witness function and a feature importance distribution across the features that optimizes a cost function that is evaluated based on the correctly paired and alternatively paired samples of the dataset. The system selects one or more features based on the computed feature importance distribution.