The recent discovery of gravitational waves (GW) by Advanced LIGO (Laser Interferometric Gravitational-wave Observatory) has impressively launched the novel field of gravitational astronomy and allowed us to glimpse exciting objects about which we could previously only speculate. Further sensitivity improvements at the low-frequency end of the detection band of future GW observatories must rely on quantum non-demolition (QND) methods to suppress fundamental quantum fluctuations of the light fields used to readout the GW signal. Here we present a novel concept of how to turn a conventional Michelson interferometer into a QND speed-meter interferometer with coherently suppressed quantum back-action noise. We use two orthogonal polarizations of light and an optical circulator to couple them. We carry out a detailed analysis of how imperfections and optical loss influence the achievable sensitivity. We find that the proposed configuration significantly enhances the low-frequency sensitivity and increases the observable event rate of binary black-hole coalescences in the range of by a factor of up to ~300.