Conceptual soil erosion model can depict basic watershed processes yet does not require intensive data. Erosion-related parameters in the current conceptual erosion models are determined either by calibration based on long series of sediment data at watershed outlets or by referring to the related empirical parameter values. This paper presented a conceptual soil erosion model which did not need long series of sediment records at watershed outlets to calibrate parameters. The model incorporated a conceptual soil erosion module, which was based on the empirical relationship between the discharge and sediment yield observed from experimental plots, into a physically-based distributed hydrology model, WetSpa Extension. The model had been tested in three watersheds in the purple soil region in China. The results of simulated discharge and sediment at outlets as well as spatial distribution of erosion rates were satisfactory. The model was also proved to be applicable in similar watersheds nearby and useful in assessing the effect of soil and water conservation practices. This conceptual model provides a new option for areas with sparse sediment data series at watershed outlets, and it has a great potential in assisting watershed management.