We report the achievement of continuous-wave (CW)-pumped second-harmonic generation (SHG) and sum frequency generation (SFG) in a layered indium selenide (InSe)-integrated microfiber. As a result of the strong interaction between the InSe nanosheets and the evanescent field, the second-order nonlinear processes are greatly enhanced in the InSe-integrated microfiber pumped by a few milliwatt CW lasers. The experimental results reveal that the intensities of SHG and SFG are quadratic and linear dependencies with the incident pump power, respectively, which is consistent with theoretical predictions. Additionally, the SHG intensity is strongly polarization-dependent on the nonaxisymmetrical distribution of the InSe nanosheets around the microfiber, providing the possibility of the SHG-polarized manipulation. The proposed device has the potential to be integrable into all-fiber systems for nonlinear applications.