Provided is a method of manufacturing a light diffusing element by which a light diffusing element having a high haze value, having strong diffusibility, and having suppressed backscattering by virtue of having a smooth surface can be manufactured at low cost and with high productivity (within a short period of time). The method of manufacturing a light diffusing element according to one embodiment of the present invention includes: a step A of mixing light diffusing fine particles and an organic solvent to prepare a mixed liquid and to swell the light diffusing fine particles; a step B of mixing the mixed liquid and a matrix-forming material containing a precursor of a resin component and ultrafine particle components; and a step C of polymerizing the precursor of a resin component to form a matrix including a resin component and the ultrafine particle components.