Embodiments relate to updating a parameter defining a policy under a Markov decision process system environment. An aspect includes updating the policy parameter stored in a storage section of a controller according to an update equation. The update equation includes a term for decreasing a weighted sum of expected hitting times over a first state (s) and a second state (s′) of a statistic on the number of steps required to make a first state transition from the first state (s) to the second state (s′).