Donor-acceptor Stenhouse adducts (DASAs) are negative photochromes that hold great promise for a variety of applications. Key to optimizing their switching properties is a detailed understanding of the photoswitching mechanism, which, as yet, is absent. Here we characterize the actinic step of DASA-photoswitching and its key intermediate, which was studied using a combination of ultrafast visible and IR pump-probe spectroscopies and TD-DFT calculations. Comparison of the time-resolved IR spectra with DFT computations allowed to unambiguously identify the structure of the intermediate, confirming that light absorption induces a sequential reaction path in which a Z-E photoisomerization of C2-C3 is followed by a rotation around C3-C4 and a subsequent thermal cyclization step. First and second-generation DASAs share a common photoisomerization mechanism in chlorinated solvents with notable differences in kinetics and lifetimes of the excited states. The photogenerated intermediate of the second-generation DASA was photo-accumulated at low temperature and probed with time-resolved spectroscopy, demonstrating the photoreversibility of the isomerization process. Taken together, these results provide a detailed picture of the DASA isomerization pathway on a molecular level.