Adherence to behavioral weight loss strategies is important for weight loss success. We aimed to examine the reliability and validity of a newly developed compliance praxis-diet (COMPASS-diet) survey with participants in a 10-week dietary intervention program. During the third of five sessions, participants of the "slim-without-diet" weight loss program (n = 253) completed the COMPASS-diet survey and provided data on demographic and clinical characteristics, and general self-efficacy. Group facilitators completed the COMPASS-diet-other scale estimating participants' likely adherence from their perspective. We calculated internal consistency, convergent validity, and predictive value for objectively measured weight loss. Mean COMPASS-diet-self score was 82.4 (SD 14.2) and COMPASS-diet-other score 80.9 (SD 13.6) (possible range 12-108), with lowest scores in the normative behavior subscale. Cronbach alpha scores of the COMPASS-diet-self and -other scale were good (0.82 and 0.78, respectively). COMPASS-diet-self scores (r = 0.31) correlated more highly with general self-efficacy compared to COMPASS-diet-other scores (r = 0.04) providing evidence for validity. In multivariable analysis adjusted for age and gender, both the COMPASS-diet-self (F = 10.8, p < 0.001, r² = 0.23) and other (F = 5.5, p < 0.001, r² = 0.19) scales were significantly associated with weight loss achieved at program conclusion. COMPASS-diet surveys will allow group facilitators or trainers to identify patients who need additional support for optimal weight loss.