Physical therapy is important for the treatment and prevention of musculoskeletal injuries, as well as recovery from surgery. In this paper, we explore techniques for automatically determining whether an exercise was performed correctly or not, based on camera images and wearable sensors. Classifiers were tested on data collected from 30 patients during normally-scheduled physical therapy appointments. We considered two lower limb exercises, and asked how well classifiers could generalize to the assessment of individuals for whom no prior data were available. We found that our classifiers performed well relative to several metrics (mean accuracy: 0.76, specificity: 0.90), but often returned low sensitivity (mean: 0.34). For one of the two exercises considered, these classifiers compared favorably with human performance.