A flexible security system has been created that allows for fluid security operations that adapt to the dynamic nature of user behavior while also allowing the security related operations themselves to be dynamic. This flexible system includes ongoing collection and/or updating of multi-perspective “security contexts” per actor and facilitating consumption of these multi-perspective security contexts for security related operations on the users. These security related operations can include policy-based security enforcement and inspection. A security platform component or security entity uses a multi-perspective security context for a user or actor. Aggregating and maintaining behavioral information into a data structure for an actor over time from different sources allows a security platform component or entity to have historical context for an actor from one or more security perspectives. Descriptors that form a security context can originate from various sources having visibility of user behavior and/or user attributes.