A method and system for generating highlights from scored data streams is disclosed. According to one embodiment, media content containing time-stamped payloads is received in a data stream from one or more sources. The received media content is recorded in a buffer for a predetermined period of time and sliced into data buckets. For each of the data buckets, a score is assigned according to user-provided keywords. The data buckets are recorded along with the assigned score to generate highlights from the one or more sources using the scores of the recorded data buckets.