A system and method for more accurately generating the surface normal calibration maps φ(AoLP) and θ(DoLP,φ) to compensate for structured scene reflections for 3-D polarimetric imaging. This is accomplished using a microfacet scattering model to develop the functional form of a polarized bidirectional reflectance distribution function (BRDF) of the object surface. The ambient radiance is ray traced to the BRDF to create the calibration maps φ(AoLP) and θ(DoLP,φ), which may be combined into a single calibration map θ(DoLP, AoLP). These maps are applied to the AoLP and DoLP images to compute an array of surface normals, which are then mapped to form a 3-D image of the object.