A method and a device for processing an intra prediction mode based video are disclosed. More specifically, a method for processing a video based on an intra-prediction mode may include determining a left prediction mode and an upper prediction mode using a prediction mode of a block neighboring a current block, wherein the left prediction mode and the upper prediction mode are determined based on a split structure of the block neighboring the current block; constructing a most probable mode (MPM) candidate list of the current block using the left prediction mode and the upper prediction mode; determining a prediction mode of the current block using a MPM index indicating a specific prediction mode in the MPM candidate list; and generating a prediction block of the current block using the prediction mode of the current block.