Systems and methods for determining a user engagement level for a content item are provided. In some aspects, indicia of one or more user interactions with a content item are received. Each user interaction in the one or more user interactions has an associated time and an interaction type. A user engagement level for the content item is determined based on the one or more user interactions, the associated times, and the interaction types. The user engagement level for the content item is stored in association with the content item.