Phytochromes, which regulate many biological processes in plants, bacteria, and fungi, can exist in two stable states, Pr and Pfr, that can be interconverted by light, via a number of intermediates such as meta-Rc. Herein we employ FTIR spectroscopy to study the Pr-to-Pfr conversion of the bacteriophytochrome Agp1 from Agrobacterium tumefaciens. Static FTIR Pfr/Pr and meta-Rc/Pr difference spectra are disentangled in terms of cofactor and protein structural changes. Guided by DFT calculations on cofactor models, the chromophore conformational changes can be grouped into structural adjustments of the cofactor-protein interactions localized in the C-D dipyrrole moiety, that is, the photoisomerisation site, and in the A-B dipyrrole moiety including the protein attachment site. Whereas changes at the C and D rings appear to be largely completed in the meta-Rc state, the structural changes in the A-B unit occur during the transition from meta-Rc to Pfr, concomitant with the main protein structural changes, as demonstrated by static and time-resolved FTIR difference spectroscopy. We employ this technique to monitor, for the first time, the dynamics of the photocycle of phytochrome on the millisecond timescale. By extending the studies to genetically engineered protein variants of Agp1, we further demonstrate that H250 and D197 as well as the PHY domain are essential for formation of the Pfr state. Based on the IR spectroscopic and available crystallographic data we discuss the role of critical amino acid residues for the protein-cofactor interactions during the photoinduced reaction cycle.