In order to investigate and realize automatic classification of cultured pearls, the authors propose a classifying method for hue and saturation of pearl. First, the pearl image was processed after transforming it from RGB color model to HIS color model. According to the histogram of I(Intensity) weight, the brightness area by global-threshold segmentation was extracted and the average values of H(Hue) and S(saturation) after decoration were gained. Aiming at the standard of hue classifying, a kind of artificial neural network method based on RPROP algorithm was adopted. Aiming at the depth difference of all kinds of colors, K-mean clustering method was employed to classify the average value of S. In this way, cultured pearls can first be classified according to the surface color of pearl, and then was further classified according to the saturation of pearls in the same color series, and finally classification of cultured pearls according to the standard of color can be achieved.