Light propagation in all-dielectric rod-type metamaterials is studied theoretically. The electric and magnetic dipole moments of the rods are derived analytically in the long-wavelength limit. The effective permittivity and permeability of a square lattice of rods are calculated by homogenizing the corresponding array of dipoles. The role of dipole resonances in the optical properties of the rod array is interpreted. This structure is found to exhibit a true left-handed behavior, confirming previous experiments [L. Peng, Phys. Rev. Lett. 98, 157403 (2007)10.1103/PhysRevLett.98.157403]. A scaling analysis shows that this effect holds at optical frequencies and can be obtained by using rods made, for example, of silicon.