A pair of outer noncircular gear has widely been used to prevent severe flow pulsation in a variable-speed elliptic gear pump. The displacement is normally several times that of the circular gear pump with the same pump cavity volume, while the instantaneous flow rate is uniform. The structure can be expected to broadly apply in agriculture, petroleum, chemical industry, food, medical treatment, and transportation. However, the beating vibration and noise that occurred easily can be detrimental to the performance of a pump, because of the special internal excitation of non-circular gear. Therefore, it is necessary to explore the nonlinear dynamics of the pump, and thereby reveal the mechanism of a rattle for better design of a high-quality elliptical gear pump with variable speed. In this study, a transmission ratio function of a two-stage non-circular gear mechanism was constructed in a variable-speed elliptical gear pump using the flow pulsation suppression of an elliptic gear pump driven by external non-circular gear. A nonlinear rattle vibration model was established in a variable-speed elliptical gear pump using the separation of elastic rotating angle considering the elastic deformation of the teeth, the static transmission error, the backlash between teeth, and the periodic load. A Runge-Kutta method was utilized to calculate the dynamic responses for the vibration curve, excitation composition, and amplitude at different rotate speeds. A systematic analysis was made on the evolution in the rattle state and system intensity, as well as the influences of pump port pressure, transmission error, and eccentricity on the rattle threshold rotation speed. The results showed that the vibration of the internal rotor was greater than that of external non-circular gear in a variable-speed elliptical gear pump. Moreover, the vibration of two-stage non-circular gears contained the time-varying instantaneous center excitation frequency, the tooth meshing excitation frequency, the multiplication, difference, and sum of these frequencies. The two-stage non-circular gears successively experienced the states of no impact, unilateral impact and bilateral impact with the increase of input rotation speed. Compared with the outer noncircular gears 1 and 2, the internal rotors 3 and 4 vibrated more violently, and entered the rattle state earlier. The dynamic meshing force rose linearly and slowly, when there was no rattle. Once the rattle occurred, the dynamic meshing force rose rapidly. Improving the pump port pressure or the manufacturing accuracy of gears can improve the rattle threshold rotation speed of a variable-speed elliptical gear pump. Among the two pairs of non-circular gears, the internal rotors 3 and 4 had a greater influence on the rattle threshold rotation speed. The rattle threshold rotation speed increased slowly and then decreased rapidly, as the rotor eccentricity advanced. Therefore, the flow rate of a pump cannot be improved, although the eccentricity can contribute to the pump displacement, due mainly to the reduction of rattle threshold rotation speed.