Systems and methods are described to facilitate generation of access policies for a network-accessible service. An authorization service may use access policies to control whether requests to access a service are authorized. A user may submit to the authorization service a request to implement a “shadow” policy, to be compared to a currently in-force policy for a service during a specified period of time. During that period, the authorization service can evaluate requests to access the service under both the currently in-force policy for the service and the shadow policy. The user can then be notified of any requests for which different authorization results are given under the currently in-force policy and the shadow policy, thus enabling the user to verify that differences between the currently in-force policy and the shadow policy are intentional rather than the result of errors within the shadow policy or currently in-force policy.