RIM: A random item mixture model to detect differential item functioning.

[en] In this article we present a new methodology for detecting Di erential Item Functioning (DIF). We introduce a DIF model, called the Random Item Mixture (RIM), that is based on a Rasch model with random item difficulties (besides the common random person abilities). In addition, a mixture model is assumed for the item difficulties such that the items may belong to one of two classes: a DIF or a non-DIF class. The crucial di fference between the DIF class and the non-DIF class is that the item difficulties in the DIF class may di ffer according to the observed person groups while they are equal across the person groups for the items from the non-DIF class. Statistical inference for the RIM is carried out in a Bayesian framework. The performance of the RIM is evaluated using a simulation study in which it is compared with traditional procedures, like the Likelihood Ratio test, the Mantel-Haenszel procedure and the standardized p-DIF procedure. In this comparison, the RIM performs better than the other methods. Finally, the usefulness of the model is also demonstrated on a real life dataset.

Disciplines :

Mathematics

Author, co-author :

Frederickx, Sofie

Tuerlinckx, Francis

De Boeck, Paul

Magis, David ; Université de Liège - ULiège > Département de mathématique > Statistique mathématique

Language :

English

Title :

RIM: A random item mixture model to detect differential item functioning.

Alternative titles :

[fr] RIM: un modèle de mixture à items aléatoires pour détecter le fonctionnement différentiel d'items

Publication date :

2010

Journal title :

Journal of Educational Measurement

ISSN :

0022-0655

Publisher :

Blackwell Publishing

Volume :

Issue :

Pages :

432-457

Peer reviewed :

Peer Reviewed verified by ORBi

Available on ORBi :

since 18 June 2010

Statistics

Number of views

73 (3 by ULiège)

Number of downloads

3 (1 by ULiège)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

Bibliography

Ackerman, T. A. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67-91.
Bolt, D. M., & Cohen, A. S. (2005). A mixture model analysis of differential item functioning. Journal of Educational Measurement, 42, 133-148.
Bolt, D. M., Cohen, A. S., & Wollack, J. A. (2001). A mixture item response model for multiple-choice data. Journal of Educational and Behavioral Statistics, 26, 381-410.
Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153-168.
Brennan, R. L. (2001). Generalizability theory. New York, NY Springer-Verlag.
Brooks, S. P., & Gelman, A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434-455.
Candell, G. L., & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12, 253-260.
Chaimongkol, S., Huffer, F. W., & Kamata, A. (2006). A Bayesian approach for fitting a random effect differential item functioning across group units. Thailand Statistician, 4, 27-41.
Chaimongkol, S., Huffer, F. W., & Kamata, A. (2007). An explanatory differential item functioning (DIF) model by the WinBUGS 1.4. Songklanakarin Journal of Science and Technology, 29, 449-458.
Clauser, B. E., Mazor, K. M., & Hambleton, R. K. (1993). The effects of purification of the matching criterion on the identification of DIF using the Mantel-Haenszel procedure. Applied Measurement in Education, 6, 269-279.
De Boeck, P. (2008). Random item IRT models. Psychometrika, 73, 533-559.
Dorans, N. J., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the scholastic aptitude test. Journal of Educational Measurement, 23, 355-368.
Engelhard, G., Hansche, L., & Rutledge, K. E. (1990). Accuracy of bias review judges in identifying differential item functioning on teacher certification tests. Applied Measurement in Education, 3, 347-360.
Fidalgo, A. M., Mellenbergh, G. J., & Muniz, J. (2000). Effects of amount of DIF, test length, and purification type on robustness and power of Mantel-Haenszel procedures. Methods of Psychological research, 5, 43-53.
Finch, H. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, Sibtest, and the IRT likelihood ratio. Applied Psychological Measurement, 29, 278-295.
Fischer, G., & Molenaar, I. (1995). Rasch models: Foundations and recent developments. New York, NY Springer-Verlag.
Geary, D. C., & DeSoto, M. C. (2001). Sex differences in spatial abilities among adults from the United States and China: Implications for evolutionary theory. Evolution and Cognition, 7, 172-176.
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2004). Bayesian data analysis (2nded.). New York, NY Chapman & Hall.
Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge, England: Cambridge University Press.
Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences (with discussion). Statistical Science, 7, 457-472.
Glas, C. A. W., & van der Linden, W. J. (2001). Modelling variability in item parameters in item response models (Technical Report). Enschede, the Netherlands: University of Twente.
Gonzalez, J., De Boeck, P., & Tuerlinckx, F. (2008). A double-structure equation model for three-mode data. Psychological Methods, 13, 337-353.
Gonzalez, J., Tuerlinckx, F., De Boeck, P., & Cools, R. (2006). Numerical integration in logistic-normal models. Computational Statistics and Data Analysis, 51, 1535-1548.
Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ Erlbaum.
Holland, P. W., & Wainer, H. (1993). Differential item functioning. Hillsdale, NJ Erlbaum.
Janssen, R., Tuerlinckx, F., Meulders, M., & De Boeck, P. (2000). A hierarchical IRT model for criterion-referenced measurement. Journal of Educational and Behavioral Statistics, 25, 285-306.
Kamata, A., Bauer, D. J., & Miyazaki, Y. (2008). Multilevel measurement modeling. In A. O'Connell & D. McCoach (Eds.), Multilevel modeling of educational data (pp. 345-388). Charlotte, NC: Information Age.
Lindley, D. V., & Novick, M. R. (1981). The role of exchangeability in inference. Annals of Statistics, 9, 45-58.
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS-a Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325-337.
Maxwell, S. E., & Delaney, H. D. (2004). Designing experiments and analyzing data: A model comparison perspective. Belmont, CA: Wadsworth.
Mellenbergh, G. J. (1982). Contingency table models for assessing item bias. Journal of Educational Statistics, 7, 105-118.
Navas-Ara, M. J., & Gomez-Benito, J. (2002). Effects of ability scale purification on the identification of dif. European Journal of Psychological Assessment, 19, 9-15.
Plake, B. S. (1980). A comparison of a statistical and subjective procedure to ascertain item validity: One step in the test validation process. Educational and Psychological Measurement, 40, 397-404.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: The Danish Institute for Educational Research.
Rost, J. (1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271-282.
Sandoval, J., & Miille, M. E. (1980). Accuracy of judgments on WISC-R item difficulty for minority groups. Journal of Consulting and Clinical Psychology, 48, 249-253.
Snijders, T. A. B. (2005). Models for longitudinal network data. In P. Carrington, J. Scott, & S. Wasserman (Eds.), Models and methods in social network analysis (pp. 215-215). New York, NY Cambridge University Press.
Soares, T. M., Goncalves, F. B., & Gamerman, D. (2009). An integrated Bayesian model for DIF analysis. Journal of Educational and Behavioral Statistics, 34, 348-377.
Tanner, M. A. (1996). Tools for statistical inference: Methods for the exploration of posterior distributions and likelihood functions. New York, NY Springer-Verlag.
Thissen, D., Steinberg, L., & Gerrard, M. (1986). Beyond group mean differences: The concept of item bias. Psychological Bulletin, 99, 118-128.
Thissen, D., Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group difference in trace lines. In H. Wainer & H. Braun (Eds.), Test validity (pp. 147-169). Hillsdale, NJ Erlbaum.
Tuerlinckx, F., & De Boeck, P. (2001). The effect of ignoring item interactions on the estimated discrimination parameters in item response theory. Psychological Methods, 6, 181-195.
Van den Noortgate, W., De Boeck, P., & Meulders, M. (2003). Cross-classification multilevel logistic models in psychometrics. Journal of Educational and Behavioral Statistics, 28, 369-386.
Voyer, D., Voyer, S., & Bryden, M. P. (1995). Magnitude of sex differences in spatial abilities: A meta-analysis and consideration of critical variables. Psychological Bulletin, 117, 250-270.
Wainer, H., & Kiely, G. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185-201.
Wainer, H., & Thissen, D. (1996). How is reliability related to the quality of test scores? What is the effect of local dependence on reliability?Educational Measurement: Issues and Practice, 15(1), 22-29.
Wang, W. C., & Su, Y. H. (2004). Effects of average signed area between two item characteristic curves and test purification procedures on the DIF detection via the Mantel-Haenszel method. Applied Measurement in Education, 17, 113-144.
Zumbo, B. D. (2007). Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4, 223-233.