Introduction: Instructional design is an established discipline for designing educational activities for learners and is applied during the development of simulation-based healthcare educational activities. Although the iterative process is already being used with alpha/beta testing during development of the simulation, the process has not been described in detail. We sought to describe this process of design changes made during a novice faculty development course for simulation-based healthcare education where participants routinely design scenarios and conduct alpha/beta testing.
Methods: Using a mixed methods study, participant written narratives and checklist/rating scales were collected on changes made during both alpha and beta testing. Narratives were analyzed using the qualitative grounded theory approach to identify emergent themes. Checklist/rating scales were analyzed for changes made to previously identified areas and how critical these changes were to the success of the scenario.
Results: Several themes of frequent changes made during the alpha/beta testing process emerged from the participant narratives that included REALISM, PROTOCOLS, INTRODUCTIONS and ROLES. The quantitative analysis of potential areas for change correlated with the emergent themes.
Conclusions: Novice scenario designers make similar changes in thematic areas during the instructional design process. Faculty development courses for novice simulation healthcare educators should anticipate attention to these areas during discussions and consider specific didactics to avoid common design pitfalls of novice educators.