Mechanisms are provided in a data processing system for utilizing algorithms based on categories in a question answering system. The mechanisms capture a history of performance and correctness metrics for identifying efficiency of respective algorithms for finding answers to questions in respective question categories in a question answering system. The mechanisms determine sets of algorithms to use for respective question categories according to efficiency and correctness analysis. The mechanisms determine a question category of a given input question and execute a set of algorithms corresponding to the question category of the given input question that meet an efficiency threshold to contribute to finding a correct answer for the given input question.