The circadian clock proteins KaiA, KaiB, and KaiC reconstitute a remarkable circa-24 h oscillation of KaiC phosphorylation that persists for many days in vitro. Here we use high-speed atomic force microscopy (HS-AFM) to visualize in real time and quantify the dynamic interactions of KaiA with KaiC on sub-second timescales. KaiA transiently interacts with KaiC, thereby stimulating KaiC autokinase activity. As KaiC becomes progressively more phosphorylated, KaiA's affinity for KaiC weakens, revealing a feedback of KaiC phosphostatus back onto the KaiA-binding events. These non-equilibrium interactions integrate high-frequency binding and unbinding events, thereby refining the period of the longer term oscillations. Moreover, this differential affinity phenomenon broadens the range of Kai protein stoichiometries that allow rhythmicity, explaining how the oscillation is resilient in an in vivo milieu that includes noise. Therefore, robustness of rhythmicity on a 24-h scale is explainable by molecular events occurring on a scale of sub-seconds.