Communication Quality Analysis (CQA) is a rigorous transcript-based coding method for assessing clinical communication quality. We compared the resource-intensive transcript-based version with a streamlined real-time version of the method with respect to feasibility, validity, reliability, and association with traditional measures of communication quality. Simulated conversations between 108 trainees and 12 standardized patients were assessed by 7 coders using the two versions of CQA (transcript and real-time). Participants also completed two traditional communication quality assessment measures. Real-time CQA was feasible and yielded fair to excellent reliability, with some caveats that can be addressed in future work. CQA ratings were moderately correlated with traditional measures of communication quality, suggesting that CQA captures different aspects of communication quality than do traditional measures. Finally, CQA did not exhibit the ceiling effects observed in the traditional measures of communication quality. We conclude that real-time CQA is a user-friendly method for assessing communication quality that has the potential for broad application in training, research, and intervention contexts and may offer improvements to traditional, self-rated communication measures.
Keywords: Communication research methods; conversation analysis; intercoder reliability; observational research methods.