Highly efficient second harmonic generation (SHG) bridging the mid-infrared (IR) and near-IR wavelengths in a coupled hyperbolic metamaterial waveguide with a nonlinear-polymer-filled nanoscale slot is theoretically investigated. By engineering the geometrical parameters, the collinear phase matching condition is satisfied between the even hybrid modes at the fundamental frequency (3,100 nm) and the second harmonic (1,550 nm). Two modes manifest the great field overlap and the significant field enhancement in the nonlinear integration area (i.e. the slot), which leads to extreme large nonlinear coupling coefficient. For a low pumping power of 100 mW, the device length is as short as 2.19 µm and the normalized conversion efficiency comes up to more than 6.37 × 10(5) W(-1)cm(-2) which outperforms that of the plasmonic-based structures. Moreover, the efficient SHG can be achieved with great phase matching tolerance, i.e., a small theoretical fabrication-error sensitivity to filling ratio and a broad pump bandwidth in a compact device length of 2.19 µm using 100 mW pump. The proposed scheme links the mature near-IR devices to the mid-IR regime and have a great potential for integrated chip-scale all-optical signal processes.