Described are systems and methods for determining an agent that performed an event within a materials handling facility. When an event is detected, an overhead image and a side-view image corresponding to the event location are obtained. Depth information is determined for the overhead image and a depth pattern generated that represents an agent. The depth pattern from the overhead image is overlaid onto the side-view image and the side-view image is segmented so that a representation of the agent included in the side-view image is extracted. A distance between the extracted representation of the agent and the event location is then determined. An association between the agent and the event may be determined based on the distance between the extracted representation and the event location.