Farmland waste has been one of the most important influencing factors on the soil environment. It is very necessary to realize an intelligent and efficient picking of farmland wastes, particularly for the high accuracy and efficiency of recognition with the simple models under complex field environments. In this study, a lightweight detection was proposed for the farmland waste under the actual field situation of the equipment using the improved yolov5s, according to the target detection and edge computing. More importantly, Artificial Intelligence (AI) was promoted in the field of smart agriculture. Firstly, some images of common wastes were collected under the complex actual field environment in the farmland. The data enhancement was then performed on the image data for the large-scale farmland wastes datasets without the over-fitting during model training. Secondly, the network unit of the classification network ShuffleNetv2 was selected to reconstruct the feature extraction network of yolov5s. The calculation and parameter amount of the model were significantly reduced to improve the running speed for the cost saving in the chip cache space. Thirdly, the convolution kernel expansion and activation function optimization were performed on the introduced lightweight network unit module, in order to effectively restore the detection accuracy of the model with less amount of model computation and parameters. Finally, the efficient intersection over union (EIoU) bounding box was introduced to reduce the target positioning error of the model in the complex environment. The reason was that there were many interference factors in the process of motion detection under the complex field environment, thus easily leading to the positioning accuracy of the target in the image. In the case of the aspect ratio for the predicted and the real frame in the loss function of complete intersection over union (CIoU), the loss item was divided into the difference between the height/width of the predicted frame and the minimum bounding frame. At the same time, the difference was gradually reduced to speed up the convergence speed and regression accuracy using the proper iteration. The experimental results show that the detection accuracy of the improved model reached 90.9% with a detection speed of 74ms/frame. Higher detection accuracy and speed of the improved model were achieved to better balance the calculation and parameter amount, compared with the current target detection of SSD and yolov3. A tradeoff was made on the performance requirements of edge computing devices for accuracy and speed. The mobile terminal was selected to verify the application of the improved model. The models before and after the improvement were deployed on the two edge computing devices (JetsonTX1 and Raspberry4B). Compared with the original, the detection speed of the improved model increased by at least 20% on the edge computing devices, indicating an excellent detection performance. The finding can provide a lightweight solution to the detection tasks of field wastes.