In one embodiment, a method includes adjusting parameters of a three-dimensional geometry corresponding to a first person to make the three-dimensional geometry represent a desired pose for the first person, accessing a neural texture encoding an appearance of the first person, generating a first rendered neural texture based on a mapping between (1) a portion of the three-dimensional geometry that is visible from a viewing direction and (2) the neural texture, generating a second rendered neural texture by processing the first rendered neural texture using a first neural network, determining normal information associated with the portion of the three-dimensional geometry that is visible from the viewing direction, and generating a rendered image for the first person in the desired pose by processing the second rendered neural texture and the normal information using a second neural network.