Sensitive and fast optical imaging is needed for scientific instruments, machine vision, and biomedical diagnostics. Many of the fundamental challenges are addressed with time stretch imaging, which has been used for ultrafast continuous imaging for a diverse range of applications, such as biomarker-free cell classification, the monitoring of laser ablation, and the inspection of flat panel displays. With frame rates exceeding a million scans per second, the firehose of data generated by the time stretch camera requires optical data compression. Warped stretch imaging technology utilizes nonuniform spectrotemporal optical operations to compress the image in a single-shot real-time fashion. Here, we present a matrix analysis method for the evaluation of these systems and quantify important design parameters and the spatial resolution. The key principles of the system include (1) time/warped stretch transformation and (2) the spatial dispersion of ultrashort optical pulse, which are traced with simple computation of ray-pulse matrix. Furthermore, a mathematical model is constructed for the simulation of imaging operations while considering the optical and electrical response of the system. The proposed analysis method was applied to an example time stretch imaging system via simulation and validated with experimental data.