Counterfactual analysis can be performed “offline”, or “after the fact”, based on data collected during a trial in which random variations are applied to the output of the system whose parameters are to be the subject of the counterfactual analysis. A weighting factor can be derived and applied to data collected during the trial to emphasize that data obtained when the random variations most closely resembled the output that would be expected if counterfactual parameters were utilized to generate the output. If the counterfactual parameters being considered differ too much from the parameters under which the trial was conducted, the offline counterfactual analysis can estimate a direction and magnitude of the change of the system performance, as opposed to deriving a specific expected system performance value. In economic transactions, the random variations can be considered variations in the price paid by another party, thereby enabling derivation of their marginal cost.