Aspects of the invention include a computer-implemented method including extracting a domain-specific object feature from a first image data, wherein the feature describes an object in the first image data. A domain-specific semantic meaning of text data is determined. The object feature is mapped to a portion of the text data, wherein the portion of the text data describes the object. A joint representation of the object and the portion of the text data is created. A second image data and a query directed towards an object in the second image data is received. An answer to the query is generated based on the joint representation.