Systems and methods that use multi-tasking and transfer learning with sparse gating mechanisms and domain knowledge to generate pheno-embeddings in a scalable manner that can improve the relevance of the patient embeddings from Electronic Health Records. A system, comprises at least one processor that executes the following computer executable components stored in memory: a structural pheno-embedding model that employs a hierarchical knowledge graph; a data augmentation component that expands on a sparse data set associated with the knowledge graph; and an embedding component that generates a specialized embedding for phenotypes using the structural pheno-embedding model and the augmented data set for a selected cohort.