A method and apparatus for reduced complexity video and image processing with special chroma handling are disclosed. Frame-type decisions are made on a video stream made up of a sequence of frames. A first subset of the frames are selected to be monochrome and generated without chroma data. A second subset of the frames are selected to be in color and generated with chroma components. In one embodiment, the first subset of frames includes odd frames and the second subset of frames includes even frames in the video stream. Under higher video frame rates, the lack of color in every other frame is not visible to the end viewer. Accordingly, subsequent processing of the output video stream permits luma-only processing of many frames in the video stream, extensively reducing the amount of computation.