An accurate and rapid identification of species is one of the most important parts of freshwater fish pre-treatment. However, deep learning-based freshwater fish classification cannot quantitatively describe the feature parameters of fish, due to the complicated feature extraction. This study aims to propose a deep learning-based fish feature point detection and freshwater fish classification. Firstly, an image acquisition device was built to acquire the images of five freshwater fish species: bream, bighead carp, grass carp, silver carp, and common carp. Secondly, the data enhancement was performed on the original images. LabelMe software was also used to label 20 feature points of the fish body, in order to construct a freshwater fish feature point detection dataset. The feature point of freshwater fish was detected using an improved AlexNet feature point detection model with the convolutional neural networks (CNN). The improved AlexNet model also adjusted the network structure of the traditional model, in order to accommodate the feature point detection task by the modified convolutional kernel size, the removal of the local response normalization layer, the addition of batch normalization layer, and the replacement of the loss function. The Euclidean distance between feature points was extracted as the feature value using the coordinates of the feature points. Feature vectors were constructed. Fisher discriminant analysis was used to identify the species of freshwater fish. A series of ablation experiments were conducted on the improved AlexNet model to clarify the impact of different improvement schemes on model performance. The results showed that the improved schemes were beneficial to improve the training efficiency and performance of the model. The average value of normalized mean error (NME) of the improved AlexNet model on the test set was 0.0099, the failure rate (FR) at thresholds of 0.02 and 0.03 were 2.50% and 0.83%, and the average detection time was 0.037 s, indicating the better accuracy and error distribution. The number of parameters and floating-point operations per second (FLOPs) of the improved AlexNet model were smaller than those of the VGG16 model. The test set detection was also better than that of the VGG16 model, compared with different feature extraction models. Although the number of parameters and FLOPs of the improved AlexNet model was larger, the average NME and the FR were lower than that of the MobileNetV2 model. The recognition accuracy of 98.0% was achieved in the freshwater fish species recognition using the model and Fisher discriminant analysis for five freshwater fish species. The average recognition time of a single image was 0.368 s. The feature point detection model can be expected to detect the feature points of five species of freshwater fish with high accuracy. The finding can also facilitate freshwater fish species identification, size detection, and fish segmentation. The freshwater fish species identification using feature point detection and Fisher's discriminant analysis can lay the foundation for the development of an automated freshwater fish classification device.