Systems and methods are provided for identifying important video frames and segments in multimedia content such as a segmented compressed domain video. Video frames in a segment are analyzed to determine intensity, contrast, and motion values for the frames and their segments. The values among frames and segments are compared to identify one or more video segments likely to be important to a viewer. The systems and methods may additionally be augmented with audio data other characteristics associated with the video frames and segments analyzed.