The automated detection of scene boundaries supports the user in browsing a video. Establishing the similarity between two respective frame sequences is made by choosing and comparing respective key-frames from each frame sequence, which are temporally as far apart from each other as possible but lie outside gradual transitions. Additionally or alternatively, film grammar- and probability-based taking into account of all types of transitions with respect to their separating and/or merging effect on the frame sequences surrounding the transitions and a differently weighted incorporation of the effects of the different types of transitions into uniting frame sequences k into scenes is provided. Thus, all types of transitions are incorporated in determining frame sequence boundaries as scene boundaries with a corresponding weighting, both their separating and merging effect is taken into account, and this knowledge is introduced on a probability basis into the decision in existing scene detection methods.