A system and method for identifying and labeling locations frequented by a user of a device, where the system and method track geographic positions and environmental or contextual factors as the user moves about, and identify locations of interest to the user via a clustering procedure. As the device collects contextual data, the system and method label each identified location to create a location model. This model allows the device to label new locations as they arise. The model may be periodically updated by separately processing geographic position data gathered after the model was created to determine if the cluster locations and labels remain accurate.