A document retrieval system tracks user selections of documents from query search results and uses the selections as proxies for manual user labeling of document relevance. The system trains a model representing the significance of different document features when calculating true document relevance for users. To factor in positional biases inherent in user selections in search results, the system learns positional bias values for different search result positions, such that the positional bias values are accounted for when computing document feature features that are used to compute true document relevance.