To overcome the shortage of massive data and multi colinearity in near infrared spectroscopy analysis, Fourier transform was employed on chestnut NIR spectra which was preprocessed with standard normal variate (SNV), and different modeling methods were utilized to improve recognition accuracy. Fourier coefficients were extracted from NIR spectra through trial method, and then discrimination models of moldy chestnuts were established based on least squares support vector machine (LSSVM). The highest mean accurate recognition rate of 93.56% was obtained when the first 35 Fourier coefficients were selected, and a hybrid algorithm, GA-LSSVM was developed and used to optimize the number of Fourier coefficients. As a result, the number of Fourier coefficients used for building recognition models was successfully reduced to 13, while the mean accurate recognition rate was raised to 97.54% and the discriminating rates of qualified chestnut, surface moldy chestnut and internal moldy chestnut were 95.89%, 100% and 98.25% for prediction, respectively. It is suggested that near infrared spectroscopy coupled with Fourier transform and GA-LSSVM methods can be used for rapid detection of moldy chestnut.