A computer-implemented method for selecting features for classification may include (1) generating a matrix X, a column vector Y, and a matrix Z from a training dataset that includes a plurality of samples with a plurality of features, (2) generating an augmented matrix from the matrix X, the column vector Y, and the matrix Z, (3) identifying one or more most-relevant features from the plurality of features by iteratively applying a sweep operation to the augmented matrix, and (4) training a classification model using the most-relevant features from the plurality of features rather than all of the plurality of features. Various other methods, systems, and computer-readable media may have similar features.