Characterization of carotid atherosclerosis and classification of plaque into symptomatic or asymptomatic and risk score estimation are of clinical value. A statistical system is described for symptomatic versus asymptomatic plaque automated classification of carotid ultrasound images and cardiovascular risk score computation. The technique is applicable for the following types of modalities for carotids: 2D Ultrasound, 3D Ultrasound, CT, MR. Wall region is segmented and features are extracted consisting of type 1 combination consisting of: (a) Higher Order Spectra; (b) Discrete Wavelet Transform (DWT); (c) Texture and (d) Wall Variability; type 2 combination consisting of: (a) Local Binary Pattern; (b) Law's Mask Energy and (c) Wall Variability and type 3 combination: (a) Trace Transform; (b) Fuzzy Grayscale Level Co-occurrence Matrix and (c) Wall Variability. These features are trained using a training classifier on training images and the coefficients are applied to on-line test patient images. The system yields the cardiovascular risk score value using the feature combinations.