We present a comprehensive, spatiotemporal, modal theory of submillimeter-wave and far-infrared power detectors. The theory is based on the contraction of the coherence tensor of the light with another coherence tensor that incorporates all of the physics of the detector. The theory is extremely general and applies to detectors of any bandwidth, with light in any state of polarization and spatiotemporal coherence. The theory applies equally to quasi-monochromatic and pulsed systems. We show that the tensor associated with the detector is a measureable quantity and outline a procedure for its experimental determination. We derive expressions for the statistical properties of a detector's output, including the correlations between the outputs of different detectors, say, in an array or interferometer. The theory provides a clear conceptual understanding of how any general detector couples to the modes of an optical system and thereby provides a powerful and flexible way of modeling the behavior of detectors and instruments.