The present invention describes a road terrain detection system that comprises a method for classifying selected locations in the environment of a vehicle based on sensory input signals such as pixel values of a camera image. The method comprises a high level spatial feature generation for selected locations in the environment called base points. The spatial feature generation of the base points is based on a value-continuous confidence representation that captures visual and physical properties of the environment, generated by so called base classifiers operating on raw sensory data. Consequently, the road terrain detection incorporates both local properties of sensor data and their spatial relationship in a two-step feature extraction process.