Rigid spherical particles in oscillating fluid flows form interesting structures as a result of fluid mediated interactions. Here we show that spheres under horizontal vibration align themselves at right angles to the oscillation and sit with a gap between them. The details of this behavior have been investigated through experiments and simulations. We have carried out experiments in which a pair of stainless steel spheres is shaken horizontally in a cell filled with glycerol-water fluid mixtures of three different viscosities, at various frequencies and amplitudes of oscillation. There is an equilibrium gap between the particles resulting from a long-range attraction and a short-range repulsion. The size of the gap was found to depend on the fluid viscosity and the vibratory parameters, and we have identified two distinct scaling regimes for the dependence of the gap on the system parameters. Using a Navier-Stokes solver the same system was simulated. The interaction force between the spheres was measured and the streaming flows induced by the motion were determined.