In this paper, we report that the angular dispersion of the output pulses in a nonlinear process can be efficiently compensated by using a cascaded prism(s) and short hollow-core fiber (HCF) configuration. Here, the prism(s) is used to suppress the angular dispersion and transform it into spatial chirp, while the HCF is used for removing this spatial chirp and the residual angular dispersion, which can also significantly improve the beam quality. The feasibility of this novel method is numerically and experimentally investigated with the ultra-broadband idler pulses centered at 1250 nm wavelength and generated by an LBO crystal based non-collinear optical parametric amplifier. The proof-of-principle experiment shows that the angular dispersion can be effectively removed and ultra-broadband idler pulses with good spectral quality and spatial profile can be obtained. The total transmission efficiency in the experiment is around 67% and the measured M x2 and M y2 can reach 1.12 and 1.04, respectively. To the best of our knowledge, this is the first reported ultra-broadband angular dispersion compensation scheme combining prism(s) and HCF, which can remarkably eliminate the angular dispersion while simultaneously possesses high efficiency, good spectral and beam spatial quality.