A system for accessing a database of a plurality of image data sets includes an acquisition unit which acquires a query for searching the database for an image data set or an image data subset comprised in an image data set. The query includes at least one medically relevant term which defines a search criteria. A determining unit determines the image data set or the image data subset included in the image data set based on the strength of semantic matches between the at least one medically relevant term and (a) corresponding medical annotation(s) describing the image data set. A retrieving unit retrieves the determined image data set or image data subset from the database.