A voice conversion system suitable for encoding small and large corpuses is disclosed. The voice conversion system comprises hardware including a neural network for generating estimated target speech data based on source speech data. The neural network includes an input layer, an output layer, and a novel split-model hidden layer. The input layer comprises a first portion and a second portion. The output layer comprises a third portion and a fourth portion. The hidden layer comprises a first subnet and a second subnet, wherein the first subnet is directly connected to the first portion of the input layer and the third portion of the output layer, and wherein the second subnet is directly connected to the second portion of the input layer and the fourth portion of the output layer. The first subnet and second subnet operate in parallel, and link to different but overlapping nodes of the input layer.