We report the results of a microscopic study of the nucleation and early growth stages of metal-catalyzed silicon nanowires in plasma-enhanced chemical vapor deposition. The nucleation of silicon nanowires is investigated as a function of different deposition conditions and metal catalysts (Sn, In and Au) using correlation of atomic force microscopy and scanning electron microscopy. This correlation method enabled us to visualize individual catalytic nanoparticles before and after the nanowire growth and identify the key parameters influencing the nanowire nucleation under plasma. The size and position of catalytic nanoparticles are found to play a significant role in the nucleation. We demonstrate that only small isolated nanoparticles in the range of 10-20 nm contribute to the nanowire growth under plasma, while larger nanoparticles are inactive because they get buried under a layer of a-Si:H before reaching supersaturation. Systematic analysis of different growth parameters reveals that the nanowire growth in plasma contradicts the vapor-liquid-solid mechanism at thermal equilibrium in many ways. The nanowire growth is much faster and proceeds even at negligible silicon solubility and bellow the eutectic temperature of the metal-silicon alloy. Based on the observations, we propose the nanowire growth under plasma to be characterized by the rapid solidification mechanism, where a crystalline silicon phase emerges from a metastable supersaturated liquid metal-silicon phase in local nonequilibrium.