Systems and methods for generating synthesized images are provided. An input medical image patch, a segmentation mask, a vector of appearance related parameters, and manipulable properties are received. A synthesized medical image patch including a synthesized nodule is generated based on the input medical image patch, the segmentation mask, the vector of appearance related parameters, and the manipulable properties using a trained object synthesis network. The synthesized nodule is synthesized according to the manipulable properties. The synthesized medical image patch is output.