To study chaos and bifurcation of a gear system, a five-degree-of-freedom nonlinear dynamic model of a gear-rotor-bearing system is established. It consists of a gear pair, supporting shafts, bearings and other auxiliary components. The effects of frequency, backlash, bearing clearance, comprehensive transmission error and stiffness on nonlinear dynamics of the system are investigated according to bifurcation diagrams, phase portraits and Poincaré maps by a numerical method. Some nonlinear phenomena such as grazing bifurcation, Hopf bifurcation, inverse-Hopf bifurcation, chaos and coexistence of attractors are investigated. Different grazing bifurcations and their causes are discussed. The critical parameters are identified, too.