To improve the accuracy of detection and classification of tomatoes with bruise, computer vision, BP algorithm and artificial neural network technology were synthetically applied to automatically identify and classify the tomatoes with bruise. First, the images of tomatoes were captured through computer vision system, then the images of tomatoes with bruise were processed applying three methods that include filtering noise and dividing images and highlighting images to identify bruise images of tomatoes applying distriction increasing. Second, multilayer forward artificial neural network trained with BP algorithms was employed to classify tomatoes with bruise. The computer vision system using the presented defect detection method and image extraction technology can save time and raise precision. The experiments show that the rate of testing precision was not less than 90%.