The invention provides an algorithm system applicable to image classification and image retrieval. The algorithm mainly comprises image modeling and and image matching modules. During image modeling, for improving characteristic description capability, a deep convolution neural network is utilized to extract depth characteristics of an image for the first time, and the hybrid Gauss model is utilized for modeling to represent the image on the basis; during image matching, for two problems of efficiency of a soil moving distance algorithm and noise sensitivity, the accurate, efficient and improved soil moving distance algorithm is proposed, Gauss noise and sparse constraints are additionally considered in an algorithm design process, moreover, through the improved designed soil moving distance algorithm, design defects of a geodesic distance measurement algorithm in the prior art are solved, after a Riemannian geometric structure of Gauss distribution is considered, three types of geodesic distance measurement modes are designed, Gauss distribution distances can be accurately measured.