Abstract: More than two million tons of tea can be produced each year in China. Tea production, consumption, and exportation have been dominated for decades in the world. However, some diseases in the growing period of tea plants have posed a serious impact on both tea yields and quality in recent years, even leading to serious economic loss. It is a high demand for the accurate identification and treatment of tea diseases for the higher production and quality. In traditional, various tea diseases can be generally identified by the accumulated experiences from experts, indicating the seriously labor-intensive and time-consuming. Since the conventional convolutional neural networks (CNN) have been ever applied to the plant disease identification, the accuracy and efficiency can drop dramatically for the small number of disease samples and uneven distribution. In this study, a transfer learning (called SE-DenseNet-FL model) was proposed for the tea disease identification of small samples. The Squeeze-and-Excitation Networks (SENet) module was first integrated into the DenseNet network structure. Among them, the SENet was used to compress the feature channels for the much more global receptive fields. As such, the SE-DenseNet-FL model was strengthened to propagate the important features for the second calibration. The Focal Loss function was then introduced to replace the loss function in the original DenseNet. Thus, the SE-DenseNet-FL model was guided to focus on the hard-to-classify samples during training, thereby alleviating the impact of the uneven sample distribution on the performance of the model. Finally, a pre-training model was obtained to pre-train with the PlantVillage dataset. A self-developed dataset of tea disease was used to fine-tune the parameters of the pre-training model through transfer learning, thereby alleviating the over-fitting from the small sample data. The dataset was taken from the organic tea garden base of Wantanhe Village, Wuyang Town, Hefeng obtains County, Enshi Tujia, and Miao Autonomous Prefecture, Hubei Province, China. A total of 2175 disease images were captured to involve the five kinds of diseases of three tea varieties, including Zhongcha No. 108, Echa No. 1, and Hefeng Dadou Tea. Five diseases included the white star disease, tea wheel spot disease, tea coal disease, tea round red spot disease, and tea leaf blight. Subsequently, an experimental operating system (CentOS) was adopted to verify the model, where a Pytorch framework was used to implement all the code and the CUDA10.0 parallel computing framework for the acceleration. A series of experiments were conducted in the field environment. Firstly, the model was optimized for the learning rate. The optimal learning rate of 0.001 was achieved, where the initial learning rates were set as 0.000 1, 0.001, 0.01, and 0.1. Then, the new transfer learning models were performed on the tea disease images. Correspondingly, transfer learning can be expected to effectively alleviate the over-fitting for the better generalization of the model. Finally, an experiment was also designed to compare with the original DenseNet model and three classic classification models as AlexNet, VGG16, and ResNet101. Consequently, the SE-DenseNet-FL model using transfer learning can be used to identify the tea diseases with an accuracy of 92.66% in the case of small samples and uneven sample distribution, indicating higher recognition accuracy and stronger robustness. The finding can provide a strong reference for the intelligent diagnosis of tea diseases.