Imagining, designing, and interpreting experiments: Using quantitative assessment to improve instruction in scientific reasoning

Abstract

Effectively teaching scientific reasoning requires an understanding of the challenges students face when learning these skills. We designed an assessment that measures undergraduate student abilities to form hypotheses, design experiments, and interpret data from experiments in cellular and molecular biology. The assessment uses intermediate-constraint free-response questions with a defined rubric to facilitate use with large classes, while identifying common reasoning errors that may prevent students from becoming proficient at designing and interpreting experiments. The assessment measured a statistically significant improvement in a senior-level biochemistry laboratory course, and a larger improvement between the biochemistry lab students and a separate cohort in a first-year introductory biology lab course. Two common errors were identified for forming hypotheses and using experimental controls. Students frequently constructed a hypothesis that was a restatement of the observation it was supposed to explain. They also often made comparisons to control conditions not included in an experiment. Both errors were most frequent among first-year students, and decreased in frequency as students completed the senior-level biochemistry lab. Further investigation of the absent controls error indicated that difficulties with reasoning about experimental controls may be widespread in undergraduate students. The assessment was a useful instrument for measuring improvement in scientific reasoning at different levels of instruction, and identified errors that can be targeted to improve instruction in the process of science.

Keywords: assessment; controls; science-process skills; scientific reasoning.