In multimedia systems requiring secure access, a method and apparatus for constructing an access control matrix for a set-top box security processor are provided. A security processor may comprise multiple security components and may support multiple user modes. For each user mode supported, at least one access rule table may be generated to indicate access rules to a security component in the security processor. An access control list comprises information regarding the access rules for a particular user mode to the security components in the security processor. An access control matrix may be generated based on the access control lists for the user modes supported by the security component. The access control matrix may be implemented and/or stored in the security processor for verifying access rights of a user mode.