Comparison of Mantel-Haenszel, Logistic Regression and Likelihood Ratio Methods to Evaluate Differential Item Functioning by Using Different Computer Software

Abstract

The aim of this study is to compare the results of differential item functioning (DIF) analysis in terms of classical test and item response theory with different computer software by using the Programme for International Student Assessment (PISA) 2012 student questionnaire data. Participants are 990 students who were randomly selected from Turkish students answering all 9 mathematics work ethics items. In terms of mathematics performance factor variable, 45% of the participants were from lower and 55% of them were from upper group. DIF was evaluated with Mantel-Haenszel, logistic regression and likelihood ratio methods. Data was analyzed by using JMETRIK, DIFAS, Zumbo SPSS Syntax, IRTLRDIF and IRTPRO software. The results showed that there was DIF in the item “homework completed in time” in terms of all analysis software and methods. Moreover, it was observed that the results of all analysis
software and methods contain similarities and differences. In the light of the findings of the study and related literature, some suggestions were made for theory and practice.