Method disclosed in the present invention is proposed to detect and eliminate flash scene in digital video. Particularly, averaging shot distribution of digital video is employed as knowledge to develop the algorithm of the present invention, and which is incorporated and used to identify three general types of shot distribution for flash scene event. In the present invention, the luminance difference between two consecutive frames is instead of actually analyzing the visual content so as to reduce the computational complexity. As a result, positions of flash frames can be exactly detected from the video signal for many applications. The method comprises the steps: frames are extracted from a video sequence inputted. A luminance difference by two adjoining frames is calculated. Then a histogram is made to record the differences, and a threshold is determined accordingly. Then the flash scene is detected and categorized into three types, and finally being eliminated.