In clinical skills training objective structured clinical examinations (OSCEs) are the method of choice to foster learning. Hence, implementing them is challenging and expensive. Thus it is investigated how an alternative procedure that keeps OSCE's essential elements, but assigns less stations to each student, works in terms of validity and justifiability.Data of n = 694 students, each taking five tasks drawn semi-randomized out of a pool of 26 tasks, strictly aligned with learning objectives, are analyzed. Despite unsurprisingly low overall reliability, a justifiable pass/fail decision is possible for 480 students (69 %). The remaining group (n = 210, 30 %) is indeed larger than with longer OSCEs, and would need ongoing assessment. The tasks' psychometric quality contributes to exams construct validity. Resource preserving short practical skills assessment is educationally valid and feasible with MedUniVienna's human medicine cohorts of n = 680.