Defining a Partial Credit Scoring Model for Multiple Response Test Items

Multiple-choice-multiple-response (MCMR) test items permit students to choose more than one response option to a given prompt. MCMR items allow researchers and instructors to gain a more nuanced measurement of students' knowledge and understanding than is available with single-response items. Several methods for scoring MCMR items have been proposed over the past several decades, but none has achieved consistent widespread use. As such, there is no standard practice for calculating scores for partially correct student responses. We present a two-part data-driven method for scoring MCMR items involving expert judgement to determine which combinations of response options should be eligible for partial credit, and quantitative analyses of students' responses to all items to determine the amount of partial credit earned by each combination. We demonstrate the efficacy of our approach by applying our method to data from the Physics Inventory of Quantitative Literacy (PIQL) collected at six universities distributed across the US. Applying our partial-credit scoring model to these PIQL data produces improved psychometric parameters for MCMR items, with negligible impact on parameter values for single-response items.