An image processing system generates 360-degree stabilized videos with higher robustness, speed, and smoothing ability using a hybrid 3D-2D stabilization model. The image processing system first receives an input video data (e.g., a 360-degree video data) for rotation stabilization. After tracking feature points through the input video data, the image processing system determines key frames and estimates rotations of key frames using a 3D reasoning based on the tracked feature points. The image processing system also optimizes inner frames between key frames using a 2D analysis based on the estimated key frame rotation. After the 3D reasoning and the 2D analysis, the image processing system may reapply a smoothed version of raw rotations to preserve desirable rotations included in the original input video data, and generates a stabilized version of the input video data (e.g., a 360-degree stabilized video).