There is provided an image processing device and an image processing method for instantaneously displaying an image of a user's field of view. An encoder encodes a celestial sphere image of a cube formed by images of multiple planes generated from omnidirectional images, the encoding being performed plane by plane at a high resolution, to generate a high-resolution encoded stream corresponding to each of the planes. The encoder further encodes, at a low resolution, the celestial sphere image to generate a low-resolution encoded stream. The present disclosure may be applied, for example, to image display systems that generate a celestial sphere image so as to display an image of the user's field of view derived therefrom.