Summary


THE IMPACT OF Q MATRIX MISSPECIFICATION ON ITEM PARAMETERS AND CLASSIFICATION ACCURACY IN THE DINA MODEL

To accurately reflect students’ current proficiency levels through achievement tests, it is essential to make determinations based on the attribute levels measured by the items. The validity of classifying students based on their measured attributes is enhanced when the items in the achievement test have varying effects on classification, taking into consideration the attributes measured by each item. The accuracy of establishing item-attribute relationships, that is, the accuracy of the Q matrix, affects the accuracy of classification, the quality of inference, and the accuracy of decisions made about students. This research aims to investigate the impact of the Q matrix misspecification in the DINA model on the item parameters and the classification of individuals in simulated data sets. Within the scope of the study, examinations were conducted regarding misspecification rates of 5%, 7.5%, and 10%, and misspecification patterns of under-fitting, over-fitting, and balanced-misfit. Simulated data were generated and analyzed using R and Mplus software. Item parameters and classifications made with misspecified Q matrices were compared with the estimations made using the true Q matrix appropriate for the data set. Under the examined conditions of this study, it was found that each misspecification condition differentiated the classifications of individuals. In the under-fitting Q matrix conditions, the classification accuracy decreased as the misspecification rate increased. In the over-fitting conditions, the classification accuracy increased as the misspecification rate increased; all resulted in lower classification accuracy compared to those obtained using the true Q matrix. In balanced-misfit conditions, an increase in the misspecification rate decreased classification accuracy. Over-fitting and balanced-misfit conditions had a more substantial negative impact on classification accuracy compared to under-fitting Q matrices. Under-fitting conditions increased the slipping parameter. Over-fitting conditions increased the guessing parameter. Balanced-misfit conditions increased under-fitting items’ slipping parameter, over-fitting items’ guessing parameter, and balanced-misfit items’ both slipping and guessing parameters. The increases in slipping parameters were much greater than the increases in guessing parameters.



Keywords

Over-fitting, under-fitting, balanced-misfit.



References