A prediction rationale analysis apparatus includes a prediction model that stores prediction models designed or trained to solve common problems, a prediction execution part that makes a prediction on the prediction target based on each prediction model to derive a model prediction result, a prediction rationale analyzer that identifies a set of prediction models based on which the model prediction result identical or approximate to the external prediction result is derived from the prediction target, and infers, as a prediction rationale for the external prediction result, a property common to the prediction models belonging to the set, and a prediction rationale difference analyzer that compares a property of any prediction model with the prediction rationale for the external prediction result to derive a prediction rationale difference indicating a difference in prediction rationale between the prediction model and the external prediction, and outputs information based on the prediction rationale difference.