Various embodiments are directed to a computer implemented method for updating a policy that is enforced by a computer program. In one embodiment, a computer communicates, to a user, data regarding one or more decisions made by the program over a period of time according to a policy. Each decision is made on the particular policy in force at the time the decision is made. Policy data for the policy is stored in a machine readable format. The user feedback data indicative of feedback by the user regarding the one or more decisions is stored. The computer identifies and ranks one or more potential variations to the policy based on a score of an objective function for each potential variation. The computer communicates, to the user, one or more suggested modifications based on the ranking of the one or more potential variations to the policy. The computer modifies the policy data based on one or more selections by the user in response to the transmission of the one or more suggested modifications.