In the process of promoting agricultural modernization, the fragmentation of cultivated landscape has seriously hindered the development of large-scale agriculture. It is of great theoretical and practical significance to scientifically reveal the spatio-temporal patterns of cultivated landscape fragmentation in China for the formulation of refined cultivated land protection policies, the efficient intensive scale use of cultivated land resources, and the modernization of agriculture and rural areas. Based on land use remote sensing data, this study calculated the patch density, edge density, landscape shape index, area-weighted mean patch area, mean patch fractal dimension index, landscape division index, splitting index, and agglomeration index of cultivated land at county-level in 1980, 1990, 2000, 2010, and 2020. The cultivated landscape pattern indices of each county were weighted and summarized to obtain the cultivated landscape fragmentation index (CFI) using principal component analysis. Kernel density analysis was then conducted to characterize the changes in the CFI across China and within each agricultural region in China. Finally, its influencing factors were analyzed with the help of geographically weighted regression model. The results of the study were showed as follows. (1) From 1980 to 2020, the average values of the CFI in China were 5.544, 5.546, 5.567, 5.619, and 5.669, respectively. The CFI was lower in the Northeast Plain, the North China Plain, the Middle and Lower Reaches of the Yangtze River, and the Sichuan Basin, while it was higher in the Loess Plateau, the Southeast Hills, the East of the Yunnan-Guizhou Plateau, the Qinling Mountains, and the Daba Mountains. The CFI in the Loess Plateau, the Southeast Hills, the eastern part of the Yunnan-Guizhou Plateau, the Qinling Mountains, and the Daba Mountains was higher. During the study period, 80.47% of the county units had an increase in the CFI. (2) The CFI in counties nationwide has been increasing as a whole, and the gap between regions in the CFI has been gradually narrowing. Except for the Qinghai Tibetan Plateau, Northern arid and semiarid regions and the Northeast China Plain, the overall trend of CFI in other regions increased from 1980 to 2020. (3) From 1980 to 2020, the county units with the decrease of cultivated land area and increased CFI accounted for the highest proportion. Most of the county units with a decrease in cultivated land were distributed south of the Hu Huanyong line, and most of the counties with an increase in cultivated land area were distributed between the first and second terraces. (4) The average elevation, distance from prefectural-level cities, and annual precipitation overall had positive effects on CFI. The effects of the percentage of construction land, population density, and GDP on CFI showed evident spatial heterogeneity, and along with socio-economic development, these factors played different roles in influencing the CFI at the different stages. In the future, China's cultivated land needs to focus on optimizing patterns. The management of cultivated landscape fragmentation needs to be based on the principle of adapting to local conditions, drawing on the management models of different regions, and achieving diversified participation, co-ordination and dialectical management.