The performance of laser-based instruments heavily depends on the stability of their laser source. Some instruments, such as the Cs-4He magnetometer, even require the frequency stabilization and the power stabilization at the same time. In this work, we design a double-locking system with a fiber-coupled output on a small bread board and apply it to the pump laser of a Cs-4He magnetometer. By carefully choosing the stabilization methods, we significantly improve the long-term simultaneous stability of frequency and power of the pump laser. The laser frequency drifts in 2 h are reduced from 100 to 10 MHz. For 10 h continuous measurements, their Allan deviation obtains about two orders of magnitude improvement for the averaging time larger than 200 s and reaches σ(τ) = 1.57 × 10-9 with a 200 s averaging time. The laser power stability for 1.8 h also obtains two orders of magnitude improvement from 3.22% to 0.031%, and its power noise reaches a level that is very close to the electronic noise of the detector. Applying this stabilization system to the pump laser of a fiber-coupled Cs-4He magnetometer, its magnetic sensor noise is significantly reduced from 0.158 to 0.009 nT, which is a reasonable noise for magnetic field detection. With this on-board design of the laser stabilization system, it is more convenient to transform the magnetometer into an outdoor device.