Etude comparative de l'indice Lz dans le cadre de données dichotomique et de données polytomique

Conference (2011, May 10)

L’indice de détection de patrons de réponses inappropriés Lz (Drasgow, Levine et Williams, 1985) est fort probablement l’indice le plus connu et le plus utilisé par les patriciens et les chercheurs en ... [more ▼]

L’indice de détection de patrons de réponses inappropriés Lz (Drasgow, Levine et Williams, 1985) est fort probablement l’indice le plus connu et le plus utilisé par les patriciens et les chercheurs en mesure. Ainsi, on a surtout étudié le comportement de Lz lorsqu’il est utilisé avec des données dichotomique mais il est aussi possible de l’analyser avec des données polytomique. Dans le cadre de cette recherche, nous proposons de comparer les résultats des indices Lz dichotomique et Lz polytomique. Pour ce faire, (i) nous utiliserons l’indice Kappa de Cohen pour comparer si les patrons de réponses détectés entre les deux versions de l’indice sont les mêmes et (ii) nous étudierons leur distribution respectives. [less ▲]

The difR package, a toolbox for the identification dichotomous differential item functioning

Conference (2011, February 25)

The purpose of this talk is to briefly introduce the R package difR to identify differential item functioning (DIF) among dichotomously scored items. The presentation is organized in three points. First ... [more ▼]

The purpose of this talk is to briefly introduce the R package difR to identify differential item functioning (DIF) among dichotomously scored items. The presentation is organized in three points. First, the general framework of DIF is outlined and the most known methods are presented succinctly. Second, the main functionalities of the difR package are described. Third, a practical application of difR is performed by a “live” analysis of a real example with several DIF methods. Future developments and objectives are discussed to conclude the talk. The difR package was jointly developed by Sébastien Béland (Université du Québec à Montréal, Canada), Francis Tuerlinckx (K. U. Leuven, Belgium) and Paul De Boeck (University of Amsterdam, The Netherlands). [less ▲]

Identification of differential item functioning in multiple-group settings: A multivariate outlier detection approach

in Multivariate Behavioral Research (2011), 46

This paper focuses on the identification of differential item functioning (DIF) when more than two groups of examinees are considered. We propose to consider items as elements of a multivariate space ... [more ▼]

This paper focuses on the identification of differential item functioning (DIF) when more than two groups of examinees are considered. We propose to consider items as elements of a multivariate space, where DIF items are outlying elements. Following this approach, the situation of multiple groups is a quite natural case. A robust statistics technique is proposed to identify DIF items as outliers in the multivariate space. For low dimensionalities, up to two three groups, also a simple graphical tool is derived. We illustrate our approach with a re-analysis of data from Kim, Cohen, and Park (1995) on using calculators for a mathematics test. [less ▲]

Beyond matching on the mean in developmental disabilities research

in Research in Developmental Disabilities (2011), 32

The matching of groups is a traditional way to control for confounding variables in developmental disability research. The equivalency of means across groups is routinely checked for these variables, but ... [more ▼]

The matching of groups is a traditional way to control for confounding variables in developmental disability research. The equivalency of means across groups is routinely checked for these variables, but not the homogeneity of their variances or the shapes of their distributions. In the present paper, it is argued that group matching can go seriously wrong unless it directly confronts the distributional concerns by the use of well-known statistical indices and very simple graphical displays of the distributions. The question of the equivalency of item response profiles is also addressed since two participants or two groups of participants can obtain the same overall score on the matching variable by passing different items. In this case, the matching cannot be considered satisfactory because of poor concordance between the molar (overall score) and molecular (item scores) levels of matching. Angoff's Delta plot method, a statistical approach for detecting differential item functioning across small groups is described. It is promising as a simple way to prove whole test/individual item correspondence and, in addition, a useful tool for making post-hoc statistical analyses at the item level on the dependant variables. [less ▲]

Des mécanismes pour assurer la validité de l’interprétation de la mesure en éducation. Tome 2: l'évaluation.

Book published by Presses de l'Université du Québec (2011)

Comparaison empirique des méthodes classiques de détection du fonctionnement différentiel d’items en psychométrie

in Raîche, Gilles; Paquette-Côté, Karine; Magis, David (Eds.) Des mécanismes pour assurer la validité de l’interprétation de la mesure en éducation. Tome 1 : la mesure. (2011)

Étude du fonctionnement différentiel d'item dans le cadre du test de classement en anglais-langue seconde (TCALS-II)

in Blais, Jean-Guy; Gilles, Jean-Luc (Eds.) Evaluation des apprentissages et technologies de l'information et de la communication : Le futur est à notre porte (2011)

Do Raven’s Colored Progressive Matrices function in the same way in typical and clinical populations? Insights from the intellectual disability field

in Intelligence (2011), 39

Standardized tests are used widely in comparative studies of clinical populations, either as dependent or control variables. Yet, one cannot always be sure that the test items measure the same constructs ... [more ▼]

Standardized tests are used widely in comparative studies of clinical populations, either as dependent or control variables. Yet, one cannot always be sure that the test items measure the same constructs in the groups under study. In the present work, 460 participants with intellectual disability of undifferentiated etiology and 488 typical children were tested using Raven’s Colored Progressive Matrices (RCPM). Data were analyzed using binomial logistic regression modeling designed to detect differential item functioning (DIF). Results showed that 12 items out of 36 function differentially between the two groups, but only 2 items exhibit at least moderate DIF. Thus, a very large majority of the items have identical discriminative power and difficulty levels across the two groups. It is concluded that RCPM can be used with confidence in studies comparing participants with and without intellectual disability. In addition, it is suggested that methods for investigating internal bias of tests used in cross-cultural, cross-linguistic or cross gender comparisons should also be regularly employed in studies of clinical populations, particularly in the field of developmental disability, to show the absence of systematic measurement error (i.e. DIF) affecting item responses. [less ▲]

La correction d'indices de détection de patrons de réponses inappropriés paramétriques: une étude de type Monte Carlo

Conference (2010, November)

L’une des conditions à respecter afin d’interpréter adéquatement les scores obtenus à un test est de s'assurer que les sujets ont répondu selon leur habileté réelle. Or, il se peut que certains individus ... [more ▼]

L’une des conditions à respecter afin d’interpréter adéquatement les scores obtenus à un test est de s'assurer que les sujets ont répondu selon leur habileté réelle. Or, il se peut que certains individus tentent de se surclasser en trichant ou de se sous-classer intentionnellement. Par chance, de nombreuses méthodes ont été développées pour détecter les individus qui adoptent de telles stratégies. Dans le cadre de cette présentation, nous nous intéresserons plus particulièrement aux indices de détection de patrons de réponses inappropriés. L’utilisation des indices de détection paramétriques comporte de nombreuses limites (Meijer et Sijtsma, 2001). Par exemple, nous savons que plusieurs indices ne peuvent pas être interprétés à partir des quantiles d’une loi de distribution connue. De plus, il est connu que l’utilisation d’une estimation de l’habileté peut grandement affecter la distribution connue de certains indices. Afin de dépasser les problèmes précédents, Snijders (2001) a développé une approche permettant de corriger les indices de détection de patrons de réponses inappropriés lorsqu'une estimation de l’habileté est considérée. Les indices ainsi corrigés suivent asymptotiquement une loi normale standard, ce qui permet la définition de seuils de détection de patrons inappropriés. Nous présenterons comment cette correction permet d'améliorer l'identification des patrons de réponses problématiques, notamment grâce à une diminution de l'écart entre l’erreur de type I empirique et le niveau de signification nominal. Une approche par simulation de données de type Monte Carlo sera présentée, et différents estimateurs de l'habileté (maximum de vraisemblance, maximum de vraisemblance a posteriori, maximum de vraisemblance pondérée) seront utilisés. [less ▲]

Identification of differential item functioning by logistic regression: application to intellectual disability data

Conference (2010, October 14)

The purpose of this talk is to briefly present the logistic regression procedure to identify differential itm functioning (DIF) in psychometrics, and to illustrate the method on a practical data set about ... [more ▼]

The purpose of this talk is to briefly present the logistic regression procedure to identify differential itm functioning (DIF) in psychometrics, and to illustrate the method on a practical data set about intellectual disability. DIF is an important research fi eld and many statistical methods have been developed in this framework. Logistic regression has the asset of not being based on item response models, and it can detect both uniform and nonuniform DIF. In addition, it can be applied to the comparison of more than two groups. This method is illustrated by comparing two groups of French pupils, one group of normal ability students and one group with intellectually disabled students. Raven's Colored Progressive Matrices test is assigned and DIF is investigated between the two groups. It is concluded that none of the items can be flagged as functioning differently. [less ▲]