The College of Intensive Care Medicine of Australia and New Zealand is responsible for credentialling trainees for specialist practice in intensive care medicine for the safety of patients and the community. This involves defining trainees' performance standards and testing trainees against those standards to ensure safe practice. The second part examination performed towards the end of the training program is a high-stakes assessment. The two clinical "Hot Cases" performed in the examination have a low pass rate, with most candidates failing at least one of the cases. There is increasing expectation for medical specialist training colleges to provide fair and transparent assessment processes to enable defensible decisions regarding trainee progression. Examinations are a surrogate marker of clinical performance with advantages, disadvantages and inevitable compromises. This article evaluates the Hot Case examination using Kane's validity framework and van der Vleuten's utility equation, and identifies issues with validity and reliability which could be managed through an ongoing improvement process.
© 2022 College of Intensive Care Medicine of Australia and New Zealand.