A method of operation of a navigation system includes: generating a training data from a randomly sampled uncategorized point of interest; generating a trained classifier model by training a classifier model using the training data; generating a category identifier, a confidence score, or a combination thereof for an uncategorized point of interest using the trained classifier model; generating a categorized point of interest by assigning the category identifier to the uncategorized point of interest; calculating a weighted confidence score based on a weighted F-measure for the category identifier, a pair of the category identifier and the confidence score; and consolidating the categorized point of interest based on the weighted confidence score for the category identifier being meeting or exceeding a threshold for displaying on a device.