Abstract: Illumination changes and shadows are the problems that must be considered in the recognition of fruits in nature scenes. In order to improve the accuracy of apple extraction under the influence of shadows, an apple surface shadow removal algorithm based on a fuzzy set was presented. In this algorithm, the image including shadows could be seen as a fuzzy matrix. The membership function was used for image de-blurring processing so as to enhance the image and then weaken the shadow's influence on apple segmentation. After the usage of a fuzzy set theory, the saturation of each pixel should be enhanced so as to reduce the difference of the adjacent pixel points. In order to verify the validity of the algorithm, a gray threshold algorithm and k-means color clustering algorithm were adapted to segment targets before and after shadow removal. In addition, the gray threshold method used in this paper was an Otsu adaptive threshold which could be objective for the judgment of segmentation results. For the k-means method used in this paper, the parameter k was selected as k=3, which means that all the images were clustered into leaves, branches, and apples. In this paper, four criteria such as Af (Segmentation error), FPR (False Positive Rate), FNR (False Negative Rate), and OI (Overlap Index) were used to evaluate the segmentations results. The results showed that after shadow removal, the target extraction area using the two segmentation algorithms were larger than that before shadow removal. The average segmentation error was 3.08% and 3.46% of the two segmentation algorithms, and it decreased 20.53% and 25.92% respectively when compared with the result before shadow removal. The FPR and FNR decreased 29.79%, 28.98%, 21.25%, and 29.83%, and OI increased 30.96% and 24.55%. In order to further verify the validity of this algorithm, the proposed algorithm was compared with a gray scale transform method and histogram equalization method. The experimental results showed that under the method proposed in this paper, the average segmentation error was reduced by 29.23% compared with the result obtained by the gray scale transform method, FPR and FNR was reduced by 30.97% and 20.40%, while OI increased by 26.60%. Then the method proposed by this paper was compared with the histogram equalization method, resulting in the segmentation errors being reduced by 25.59%, and FPR and FNR was reduced by 22.74% and 27.56%, while OI increased by 27.43%. This series of data showed that the presented algorithm could get better shadow removal effects. All these results showed that the presented shadow removal algorithm proposed by this paper could improve the target segmentation performance and are feasible and effective to remove the shadows of apples. This method has very broad further significance.