Red Imported Fire Ant (RIFA, Solenopsis invicta Buren) is one of the worst invasive species, particularly for the target of a national cost-shared eradication program. Rapid and accurate detection of RIFA has been highly urgent for the unmanned monitoring of the epidemic situation in recent years. In this study, an improved YOLOv5s model was proposed to detect and recognize the RIFA insects and nests using image processing. The specific procedure was as follows. Firstly, the image data of the RIFA nest was collected in the epidemic area of RIFA on the lawn, where the resolution ratio was compressed to 640×480 pixels. Data collection was carried out on the images of RIFA insects on a single RIFA nest. A total of 1400 images were collected as a data set. Labelling software was used to manually label the RIFA insects in the images. Specifically, the training label map was obtained using the minimum circumscribed rectangle containing all pixels of the RIFA insect. The labelled image data set was randomly divided into the training, test, and verification set, according to 8:1:1. Secondly, a compression operation was performed on the background area of the value component image of the RIFA nest in the lawn environment. The OTSU threshold was used to segment the difference images between the compressed and hue component. A super green model was selected to extract the difference that fused with the background area. An attention mechanism module was added to the backbone network of the YOLOv5s model. An improved YOLOv5s network was obtained to detect and recognize the RIFA insects. The reason was that the attention mechanism was commonly used to calibrate the channel and spatial location for the better performance of the improved model. Finally, the collected RIFA nests and RIFA insect images were tested on the detection and recognition model. The results showed that the highest Intersection over Union (IoU) of RIFA nest images in a lawn environment was 96.87%, where the samples with IoU higher than 90% accounted for 25%, the samples with IoU between 80%-90% accounted for 56.67%, the samples with IoU lower than 80% accounted for 18.33%, and the samples with IoU higher than 80% accounted for more than 81%. Additionally, the average detection speed of RIFA insect images reached 48.53 frames/s, the precision was 91.50%, the recall rate was 89.28%, the Average Precision (AP) was 95.40%, and the F1 comprehensive evaluation index was 90.38%. The performance of the imported YOLOv5s model was greatly improved, compared with the original. The finding can provide a strong reference for the intelligent detection of RIFAs in a lawn environment.