We study the elasticity of random fiber networks. Starting from a microscopic picture of the nonaffine deformation fields, we calculate the macroscopic elastic moduli both in a scaling theory and a self-consistent effective medium theory. By relating nonaffinity to the low-energy excitations of the network ("floppy modes"), we achieve a detailed characterization of the nonaffine deformations present in fibrous networks.