Abstract: The segmentation and localization of kiwi fruit in the field under nature scenes are the key technology for realizing automatic kiwi fruit-picking. Recently, the research studies about fruit recognition have been limited to a single color space and considered less about complex backgrounds in a field, resulting in a low recognition rate. In order to improve the recognition effect of kiwifruit in the field, a method based on an Adaboost algorithm was developed for segmentation between kiwi fruit and its background. First, it needs to extract and optimize the effective features. For ascertaining the color features between kiwi fruit and its background, three commonly used color spaces such as RGB, HSV and La*b* were chosen, and the 100 points of kiwi fruit and 200 sample points of background in the collected images were analyzed. It was concluded that the G-R channel in RGB color space, the horizontal distance index, the a*-b* channel and a* channel in La*b* color space, H-S channel and H channel in HSV color space can separate kiwi and its background correctly. The study then expounded the principle of an Adaboost algorithm, and used these six channels to build six different weak classifiers. Next, 300 kiwi fruit and background sample points were used for training, selecting a weak classifier automatically in training, and finally a stage classifier was generated after four iterations. After that, 655 sample points containing 208 points of kiwi fruit and 477 sample points of background selected as test samples were tested for precision validation. Both the classification precision of stage classifier being 94.20% and the coefficient of Kappa being 0.88 were higher than any of the weak classifiers'. The recognition experiment was conducted to test the algorithm with 215 kiwifruit taken from 80 sample images. The rate of fault recognition was 7.0%, the rate of missed recognition was only 3.7%, and the rate of successful recognition reached 96.7%. The rate of successful recognition being 92.1% and 88.9% were lower than this paper method when the R-G channel and horizontal distance index were used. The test results showed a R-G channel could not get a correct segmentation between soil and kiwifruit, and twigs and stalks easily resulted in false segmentation. The horizontal distance index could remove the trunks accurately, but leaves easily resulted in false segmentation. Both these two methods had their identification advantages, but they sometimes wrongly segmented the background. The Adaboost algorithm compromised the identifying strengths of the two methods to make up for its shortcomings, and it achieved an ideal effect for the segmentation between kiwifruit and trunk, soil and branches. Finally, the kiwifruit recognition based Adaboost algorithm had preferable performance because of its restraining the influence of a complex background such as the sky and the earth's surface effectively. This method was feasible and valid for kiwi fruit recognition in a field and with high recognition accuracy. This paper provided a technical basis for the development of a kiwifruit picking robot.