A method of training a generator G of a Generative Adversarial Network (GAN) includes receiving, by an encoder E, a target data Y; receiving, by the encoder E, an output G(Z) of the generator G, where the generator G generates the output G(Z) in response to receiving a random sample Z and where a discriminator D of the GAN is trained to distinguish which of the G(Z) and the target data Y; training the encoder E to minimize a difference between a first latent space representation E(G(Z)) of the output G(Z) and a second latent space representation E(Y) of the target data Y, where the output G(Z) and the target data Y are input to the encoder E; and using the first latent space representation E(G(Z)) and the second latent space representation E(Y) to constrain the training of the generator G.