Reference : Two issues in differential item functioning and two recently suggested solutions
Scientific congresses and symposiums : Unpublished conference
Physical, chemical, mathematical & earth Sciences : Mathematics
Two issues in differential item functioning and two recently suggested solutions
[fr] Deux problèmes de fontionnement différentiel d'items et deux solutions récentes
Magis, David mailto [Université de Liège - ULg > Département de mathématique > Statistique mathématique >]
Facon, Bruno [> >]
De Boeck, Paul [> >]
8th Conference of the International Test Commission
3-5 juillet 2012
International Test Commission
[en] Two issues of current interest in the framework of differential item functioning (DIF) are considered. First, in the presence of small samples of respondents, the asymptotic validity of most traditional DIF detection methods is not guaranteed. Second, even with large samples of respondents, test score based methods such as Mantel-Haenszel) are affected by Type I error inflation when the true underlying model is not the Rasch model and in the presence of impact.
To deal with small samples of respondents, Angoff’s Delta plot may be considered as a simple and straightforward DIF method. An improvement is proposed, based on acceptable assumptions, to select an optimal classification threshold rather than fixing it arbitrarily (as with the standard Delta plot). This modified Delta plot was compared to its standard version and to the Mantel-Haenszel method by means of simulations. Both, Mantel-Haenszel and the modified Delta plot outperform Angoff’s proposal, but the modified Delta plot is much more accurate for small samples than Mantel-Haenszel.
For the second issue, a robust outlier approach to DIF was developed, by considering DIF items as outliers in the set of all tests items, and flagging the outliers with robust statistical inferential tools. This approach was compared with the Mantel-Haenszel method using simulations. Stable and correct Type I errors are observed for the robust outlier approach, independent of the underlying model, while Type I error inflation is observed for the Mantel-Haenszel method. The robust outlier method may therefore be considered as a valuable alternative.

File(s) associated to this reference

Additional material(s):

File Commentary Size Access
Private access
ITC 2012 - David Magis.pdfSlides139.82 kBRequest copy

Bookmark and Share SFX Query

All documents in ORBi are protected by a user license.