Described are techniques and systems for determining a state of activity of a user, and performing operations responsive to the state. The state may be determined using environmental data and operational data. The environmental data provides data indicative of the user's level of activity and may include data from sensors such as cameras, microphones, motion sensors, and so forth. The operational data includes data about performance of one or more services which may be associated with the user. The state may indicate that the user is resting, awake, working, exercising, and so forth. For example, while the user is inactive, such as while resting, resource-intensive operations may be performed. By determining when the user is inactive and detecting unusual levels of activity by the service problems such as damage to the service or malicious activity may be identified.