Stacking two-dimensional (2D) materials into multi-layers or heterostructures, known as van der Waals (vdW) epitaxy, is an essential degree of freedom for tuning their properties on demand. Few-layer black phosphorus (FLBP), a material with high potential for nano- and optoelectronics applications, appears to have interlayer couplings much stronger than graphene and other 2D systems. Indeed, these couplings call into question whether the stacking of FLBP can be governed only by vdW interactions, which is of crucial importance for epitaxy and property refinement. Here, we perform a theoretical investigation of the vibrational properties of FLBP, which reflect directly its interlayer coupling, by discussing six Raman-observable phonons, including three optical, one breathing and two shear modes. With increasing sample thickness, we find anomalous redshifts of the frequencies for each optical mode but a blueshift for the armchair shear mode. Our calculations also show splitting of the phonon branches, due to anomalous surface phenomena, and strong phonon-phonon coupling. By computing uniaxial stress effects, inter-atomic force constants and electron densities, we provide a compelling demonstration that these properties are the consequence of strong and highly directional interlayer interactions arising from the electronic hybridization of the lone electron-pairs of FLBP, rather than from vdW interactions. This exceptional interlayer coupling mechanism controls the stacking stability of BP layers and thus opens a new avenue beyond vdW epitaxy for understanding the design of 2D heterostructures.