In this Letter we experimentally implement a single photon Bell test based on the ideas of S. Tan et al. [Phys. Rev. Lett. 66, 252 (1991)]] and L. Hardy [Phys. Rev. Lett. 73, 2279 (1994)]]. A double homodyne measurement is used to measure correlations in the Fock space spanned by zero and one photons. Local oscillators used in the correlation measurement are distributed to two observers by copropagating it in an orthogonal polarization mode. This method eliminates the need for interferometrical stability in the setup, consequently making it a robust and scalable method.