Model analysis: Representing and assessing the dynamics of student learning

Decades of education research have shown that students can simultaneously possess alternate knowledge frameworks and that the development and use of such knowledge are context dependent. As a result of extensive qualitative research, standardized multiple-choice tests such as Force Concept Inventory and Force-Motion Concept Evaluation tests provide instructors tools to probe their students' conceptual knowledge of physics. However, many existing quantitative analysis methods often focus on a binary question of whether a student answers a question correctly or not. This greatly limits the capacity of using the standardized multiple-choice tests in assessing students' alternative knowledge. In addition, the context dependence issue, which suggests that a student may apply the correct knowledge in some situations and revert to use alternative types of knowledge in others, is often treated as random noise in current analyses. Through research, we developed a new modeling method, model analysis. It applies the results from qualitative studies to establish a quantitative representation framework, with which the space of students' knowledge and the probabilities for students to use different types of knowledge in a range of equivalent contexts can be quantitatively represented and analyzed. This provides a new method for quantitative assessment in education, which can generate much richer information than what is available from score-based analysis.