Geometric confinement strongly influences the behavior of microparticles in liquid environments. However, to date, nonspherical particle behaviors close to confining boundaries, even as simple as planar walls, remain largely unexplored. Here, we measure the height distribution and orientation of colloidal dumbbells above walls by means of digital in-line holographic microscopy. We find that while larger dumbbells are oriented almost parallel to the wall, smaller dumbbells of the same material are surprisingly oriented at preferred angles. We determine the total height-dependent force acting on the dumbbells by considering gravitational effects and electrostatic particle-wall interactions. Our modeling reveals that at specific heights both net forces and torques on the dumbbells are simultaneously below the thermal force and energy, respectively, which makes the observed orientations possible. Our results highlight the rich near-wall dynamics of nonspherical particles and can further contribute to the development of quantitative frameworks for arbitrarily shaped microparticle dynamics in confinement.