Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining causal models for controlling environments. One of the methods includes identifying a procedural instance; selecting control settings for the procedural instance, comprising, for a particular one of the controllable elements: assigning the procedural instance to a cluster for the particular controllable element in accordance with current values of a set of clustering parameters for the particular controllable element; and selecting a setting for the particular controllable element for the procedural instances based on a causal model that is specific to the cluster; obtaining environment responses to the selected control settings that define a value of the performance metric for the procedural instance; and updating, for the particular controllable element, the causal model for the cluster for the controllable element to which the procedural instance was assigned based on the value of the performance metric.