Immersion gratings have high dispersion efficiency and have important application value in miniaturized imaging spectrometers, but its serious dispersion nonlinearity causes difficulties in calibration and image processing, which limits its application range. To solve this, this paper presents a design method for a two-material linear dispersion immersion grating device design method, and a compact small F-number immersion grating spectrometer based on it. First the vector form dispersion equation of the two-material immersion grating is derived and the linear spectral dispersion immersion grating design process is given, then a compact small F-number uniform dispersion imaging spectrometer is given as a design example using the proposed method. The results show that when the operating band of the system is 1590-1675 nm, the spectral resolution is better than 0.25 nm, and F-number can achieve better than 2. Compared with traditional single-material immersion grating imaging spectrometer, the designed imaging spectrometer dispersion linearity is significantly improved. Finally, the influence of prism materials, structure parameters and grating parameters on dispersion nonlinearity is analyzed. Design and analysis results show that the proposed two-material immersion grating device has much better spectral dispersion nonlinearity correction ability, and its design method can provide reference to the compact spectrometer design based on immersion gratings.