An image display device includes: a light source that emits a beam having a spreading angle in a first direction and a spreading angle in a second direction, which is smaller than the spreading angle in the first direction, in a direction perpendicular to the first direction; an afocal optical system member that has a focal point and changes the beam emitted by the light source into a parallel beam; an anisotropic diffusion plate that is arranged at the focal point and that widens at least the spreading angle in the second direction; a vibration unit that vibrates the anisotropic diffusion plate; a fly-eye lens array on which the parallel beam emitted from the afocal optical system member becomes incident; an image display element that modulates the beam that has passed through the fly-eye lens array into image display light.