PECAP - Redesigning Legacy Conceptual Inventories: Using Differential Item Functioning to Explore Item-Level Differences Across Introductory Physics Course-Type and Students' Gender Identity

In an effort to redesign the problematic legacy conceptual inventories, we've developed and tested a new concept inventory for 1D Kinematics. We examine potential item bias between various student groups within our new 20-item assessment administered in introductory physics courses at three R1 universities over several semesters. In our preliminary analysis, we focused on item-level differences between the type of course students were enrolled in (e.g. calculus vs. algebra –based) as well as between students' gender identity. Graphical representations of item difficulty and discrimination between groups will be presented. We explored 4 different Differential Item Functioning (DIF) methods, all of which were binned and/or purified variations of the Classical Test Theory Mantel-Haenszel statistic. Results indicate that 3 items were flagged for large DIF effects; 2 of which were flagged in one of the categories favouring their respective reference groups (calculus and men), and 1 item which was flagged in both categories favouring focal groups (algebra and non-men). This suggests these particular items warrant further review to ensure the inventory is fair for all students.