The purpose of this study was to evaluate the sliced body volume (SBV) as a respiratory surrogate by comparing with the real-time position management (RPM) in phantom and patient cases. Using the SBV surrogate, breathing signals were extracted from unsorted 4D CT images of a motion phantom and 31 cancer patients (17 lung cancers, 14 abdominal cancers) and were compared to those clinically acquired using the RPM system. Correlation coefficient (R), phase difference (D), and absolute phase difference (D(A)) between the SBV-derived breathing signal and the RPM signal were calculated. 4D CT reconstructed based on the SBV surrogate (4D CT(SBV)) were compared to those clinically generated based on RPM (4D CT(RPM)). Image quality of the 4D CT were scored (S(SBV) and S(RPM), respectively) from 1 to 5 (1 is the best) by experienced evaluators. The comparisons were performed for all patients, and for the lung cancer patients and the abdominal cancer patients separately. RPM box position (P), breathing period (T), amplitude (A), period variability (V(T)), amplitude variability (V(A)), and space-dependent phase shift (F) were determined and correlated to S(SBV). The phantom study showed excellent match between the SBV-derived breathing signal and the RPM signal (R = 0.99, D= -3.0%, D(A) = 4.5%). In the patient study, the mean (± standard deviation (SD)) R, D, D(A), T, V(T), A, V(A), and F were 0.92 (± 0.05), -3.3% (± 7.5%), 11.4% (± 4.6%), 3.6 (± 0.8) s, 0.19 (± 0.10), 6.6 (± 2.8) mm, 0.20 (± 0.08), and 0.40 (± 0.18) s, respectively. Significant differences in R and D(A) (p = 0.04 and 0.001, respectively) were found between the lung cancer patients and the abdominal cancer patients. 4D CT(RPM) slightly outperformed 4D CT(SBV): the mean (± SD) S(RPM) and S(SBV) were 2.6 (± 0.6) and 2.9 (± 0.8), respectively, for all patients, 2.5 (± 0.6) and 3.1 (± 0.8), respectively, for the lung cancer patients, and 2.6 (± 0.7) and 2.8 (± 0.9), respectively, for the abdominal cancer patients. The difference between S(RPM) and S(SBV) was insignificant for the abdominal patients (p = 0.59). F correlated moderately with S(SBV) (r = 0.72). The correlation between SBV-derived breathing signal and RPM signal varied between patients and was significantly better in the abdomen than in the thorax. Space-dependent phase shift is a limiting factor of the accuracy of the SBV surrogate.