References of "Magis, David"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailDetection of differential item functioning using the lasso approach
Magis, David ULg; Tuerlinckx, Francis; De Boeck, Paul

in Journal of Educational & Behavioral Statistics (in press)

This paper proposes a novel approach to detect differential item functioning (DIF) among dichotomously scored items. Unlike standard DIF methods that perform an item-by-item analysis, we propose the “LR ... [more ▼]

This paper proposes a novel approach to detect differential item functioning (DIF) among dichotomously scored items. Unlike standard DIF methods that perform an item-by-item analysis, we propose the “LR lasso DIF method”: logistic regression (LR) model is formulated for all item responses. The model contains item specific intercepts, an effect of the sum score and item-group interaction (i.e. DIF) effects, with a lasso penalty on all DIF parameters. Optimal penalty parameter selection is investigated through several known information criteria (AIC, BIC and cross-validation) as well as through a newly developed alternative. A simulation study was conducted to compare the global performance of the suggested “LR lasso DIF” method to the logistic regression and Mantel-Haenszel methods (in terms of false alarm and hit rates). It is concluded that for small samples the LR lasso DIF approach globally outperforms the logistic regression method, and also the Mantel-Haenszel method, especially in the presence of item impact, while it yields similar results with larger samples. [less ▲]

Detailed reference viewed: 31 (3 ULg)
Full Text
Peer Reviewed
See detailA note on the equivalence between observed and expected information functions with polytomous IRT models
Magis, David ULg

in Journal of Educational & Behavioral Statistics (in press)

The purpose of this note is to study the equivalence of observed and expected (Fisher) information functions with polytomous item response theory (IRT) models. It is established that observed and expected ... [more ▼]

The purpose of this note is to study the equivalence of observed and expected (Fisher) information functions with polytomous item response theory (IRT) models. It is established that observed and expected information functions are equivalent for the class of divide-by-total models (including partial credit, generalized partial credit, rating scale and nominal response models), but not for the class of difference models (including the graded response and modified graded response models). Yet, observed information function remains positive in both classes. Straightforward connections with dichotomous IRT models and further implications are outlined. [less ▲]

Detailed reference viewed: 13 (2 ULg)
Full Text
Peer Reviewed
See detailA note on weighted likelihood and Jeffreys modal estimation of proficiency levels in polytomous item response models
Magis, David ULg

in Psychometrika (in press)

Warm (1989) established the equivalence between the so-called Jeffreys modal and the weighted likelihood estimators of proficiency level with some dichotomous item response models. The purpose of this ... [more ▼]

Warm (1989) established the equivalence between the so-called Jeffreys modal and the weighted likelihood estimators of proficiency level with some dichotomous item response models. The purpose of this note is to extend this result to polytomous item response models. First, a general condition is derived to ensure the perfect equivalence between these two estimators. Second, it is shown that this condition is fulfilled by two broad classes of polytomous models including, among others, the partial credit, rating scale, graded response and nominal response models. [less ▲]

Detailed reference viewed: 28 (5 ULg)
Full Text
Peer Reviewed
See detailPassage de l’administration fixe d’un test à une administration adaptative : application au TCALS-II
Magis, David ULg; Raîche, Gilles

in Raîche, Gilles; Ndinga, Pascal; Meunier, Hélène (Eds.) L'interdisciplinarité de la mesure et de l’évaluation (in press)

La problématique du passage de l’administration fixe (de type papier-crayon) à une administration adaptative d’un test est étudiée. Une méthode en deux étapes est présentée. Dans un premier temps, des ... [more ▼]

La problématique du passage de l’administration fixe (de type papier-crayon) à une administration adaptative d’un test est étudiée. Une méthode en deux étapes est présentée. Dans un premier temps, des patrons de réponses sont générés selon une administration fixe, dans le but de déterminer des valeurs admissibles de l’erreur-type d’estimation du niveau d’habileté. Ensuite, ces valeurs sont utilisées comme critères d’arrêt lors d’une administration adaptative du même test. La longueur du test est alors considérée pour évaluer la qualité du test par rapport à sa version fixe. Le test de classement en anglais, langue seconde, au collégial (TCALS-II) est utilisé en guise d’illustration. Il est établi qu’une administration adaptative du TCALS-II permettrait de réduire sensiblement la longueur du test, sans perte de qualité de l’estimation des niveaux d’habileté. Toutefois, cette amélioration est limitée aux sujets ne présentant pas un niveau d’habileté trop faible ou trop important. [less ▲]

Detailed reference viewed: 23 (5 ULg)
Peer Reviewed
See detailLe testing adaptatif informatisé : une brève introduction
Magis, David ULg

Conference (2015, January 27)

L’objet de cet exposé est de présenter les grands principes et concepts du testing adaptatif informatisé (TAI). Les éléments abordés sont : le TAI face au test fixe (papier-crayon), les principes généraux ... [more ▼]

L’objet de cet exposé est de présenter les grands principes et concepts du testing adaptatif informatisé (TAI). Les éléments abordés sont : le TAI face au test fixe (papier-crayon), les principes généraux du TAI (banque d’items, estimation provisoire et finale de la compétence, sélection des items à administrer, règles d’arrêt), les principes spécifiques au TAI (contrôle de l’exposition des items, équilibrage du contenu des tests). L’exposé se veut didactique et général afin de dessiner les contours du TAI. Il se termine par un état de l’art sur les recherches actuelles sur le TAI. [less ▲]

Detailed reference viewed: 12 (1 ULg)
See detailIntroduction to item response theory (IRT) and computerized adaptive testing (CAT) with the R software
Magis, David ULg

Scientific conference (2015, January 13)

Item response theory (IRT) has become an important field of research for psychology and educational assessment. Recently, with the increase of computational power, several IRT-related topics have emerged ... [more ▼]

Item response theory (IRT) has become an important field of research for psychology and educational assessment. Recently, with the increase of computational power, several IRT-related topics have emerged, among others, computerized adaptive testing (CAT). The main aim of CAT is to provide a framework for individualized assessment by means of optimal item selection and administration to the test takers. CAT has several assets to linear (non-adaptive) testing: individualized assessment, limited risk of cheating or fraud, shorter tests providing the same amount of information as longer linear tests, automatic scoring and reporting at the end of the test. Practical use of CAT, however, remains limited so far due to several factors (lack of available large item banks, content validity and security, lack of suitable software for practical CAT assessment, ethical issues in administering different tests to estimate the same ability, etc.). The purpose of this workshop is threefold: (a) to provide a general overview of IRT and CAT, (b) to introduce the R software in a user-oriented way, as well as several IRT tools (including the package catR for CAT simulations), (c) to perform practical training sessions with the participants. The workshop will be a mix of oral presentations, demonstrations related to the R software, and practical sessions where participants will be invited to train with R and catR. The R software is an open-source platform for statistical inference and testing, graphical display and data visualization. It also holds several add-on packages for specific IRT purposes (item calibration, ability estimation, multidimensional scaling, equating, differential item functioning etc.). The R community is worldwide and proposes free exchanges of shared R packages through the CRAN (comprehensive R archive network). In this workshop, the R package catR will be examined and used in the practical sessions. [less ▲]

Detailed reference viewed: 37 (1 ULg)
Full Text
See detailPsychometrics and bibliometrics: Overview of future possible interactions
Magis, David ULg

Scientific conference (2014, December 02)

Bibliometrics is a growing research field with the main aim of collecting bibliometric information (citations, publications, awards...) to infer valuable rankings or to predict future academic performance ... [more ▼]

Bibliometrics is a growing research field with the main aim of collecting bibliometric information (citations, publications, awards...) to infer valuable rankings or to predict future academic performance. However, most bibliometric approaches are based either on pairwise comparisons of "objects" (authors, funding applications, academics...) with respect to selected "criteria" (publications, citation rate, h-index...), or on modeling observable academic outcomes using observed such criteria. The purpose of this talk is twofold. First, an improvement of the pairwise comparison approach will be outlined on the basis of so-called multicriteria decision aid (MCDA) routines, which will imply an increased flexibility and added freedom for the decision process. Second, we will start a deeper reflexion on possible connections between bibliometrics and psychometrics. Among others, the possibility of modeling latent "academic ability" as outcome of interest, given manifest criteria, will be put to an open discussion and exchange with the listeners [less ▲]

Detailed reference viewed: 18 (1 ULg)
Full Text
Peer Reviewed
See detailThe sentence repetition task: A powerful diagnostic tool for French children with specific language impairment
Leclercq, Anne-Lise ULg; Quémart, Pauline; Magis, David ULg et al

in Research in Developmental Disabilities (2014), 35

This study assesses the diagnostic accuracy and construct validity of a sentence repetition task that is commonly used for the identification of French children with specific language impairment (SLI ... [more ▼]

This study assesses the diagnostic accuracy and construct validity of a sentence repetition task that is commonly used for the identification of French children with specific language impairment (SLI). Thirty-four school-aged children with a confirmed, diagnostically based diagnosis of SLI, and 34 control children matched on age and nonverbal abilities performed the sentence repetition task. Two general scoring measures took into account the verbatim repetition of the sentence and the number of words accurately repeated. Moreover, five other scoring measures were applied to their answers in order to separately take into account their respect of lexical items, functional items, syntax, verb morphology, and the general meaning of the sentence. Results show good to high levels of sensitivity and specificity at the three cut-off points for all scoring measures. A principal component analysis revealed two factors. Scoring measures for the respect of functional words, syntax and verb morphology provided the largest loadings to the first factor, while scoring measures for the respect of lexical words and general semantics provided the largest loadings to the second factor. Sentence repetition appears to be a valuable tool to identify SLI in French children, and the ability to repeat sentences correctly is supported by two factors: a morphosyntactic factor and a lexical factor. [less ▲]

Detailed reference viewed: 46 (9 ULg)
See detailA lasso penalization approach to differential item functioning
Magis, David ULg

Conference (2014, November 24)

Detailed reference viewed: 13 (2 ULg)
Full Text
Peer Reviewed
See detailOn the finiteness and uniqueness of the weighted likelihood estimator of ability in polytomous IRT models
Magis, David ULg; Verhelst, Norman

Conference (2014, November 21)

In item response theory (IRT), the weighted likelihood (WL) estimator has become a central method to estimate ability levels of respondents. Primarily introduced with dichotomous IRT models (Warm, 1989 ... [more ▼]

In item response theory (IRT), the weighted likelihood (WL) estimator has become a central method to estimate ability levels of respondents. Primarily introduced with dichotomous IRT models (Warm, 1989), it was later extended to polytomous IRT models (Samejima, 1998). However, very few information is available about the behavior of the WL estimator, and especially about the uniqueness of the ability estimates as well as their finiteness. The purpose of this talk is to establish that with polytomous item response models, the WL estimator of ability always returns finite values. This result is valid for the class of difference models and divide-by-total models, independently of the number of items and the response patterns. However, such estimates may not necessarily be unique, as the WL solving equation may provide several optimal values. Some examples are considered to illustrate both findings. [less ▲]

Detailed reference viewed: 43 (3 ULg)
Peer Reviewed
See detailRecent advances and improvements in computerized adaptive testing with the R package catR
Barrada, Juan Ramon; Magis, David ULg

Conference (2014, October 08)

The purpose of this talk is to present recent advances and developments of the R package catR. This package allows for random generation of response patterns under computerized adaptive testing (CAT) and ... [more ▼]

The purpose of this talk is to present recent advances and developments of the R package catR. This package allows for random generation of response patterns under computerized adaptive testing (CAT) and holds various options to select the first items, estimate ability, select the next item, stop the test and return final results. Two main improvements were realized. First, several rules for next item selection were added, among others, Kullback- Leibler, progressive and proportional methods. Second, catR was limited to dichotomous IRT models so far. The most recent update allows now for several polytomous IRT models, such as partial and generalized partial credit model, graded and modified graded response models, rating scale model, and nominal response model. All improvements will be shortly presented, both from a theoretical point of view and in terms of practical implementation in catR. If possible, several illustrative examples will be displayed in a short live demonstration of catR. [less ▲]

Detailed reference viewed: 56 (0 ULg)
Full Text
Peer Reviewed
See detailDetection of differential item functioning using the lasso approach
Magis, David ULg; Tuerlinckx, Francis; De Boeck, Paul

Conference (2014, July 22)

The purpose of this talk is to present a novel approach to detect differential item functioning (DIF) among dichotomously scored items. Unlike standard DIF methods that perform an item-by-item analysis ... [more ▼]

The purpose of this talk is to present a novel approach to detect differential item functioning (DIF) among dichotomously scored items. Unlike standard DIF methods that perform an item-by-item analysis, we consider a logistic regression model including item-group interaction (i.e. DIF) effects of all items simultaneously. The method is based on penalized maximum likelihood estimation of a model with a lasso penalty on all possible DIF parameters. Optimal penalty parameter selection is investigated through several known information criteria (such as AIC and BIC) as well as a newly developed weighted alternative. A simulation study was conducted to compare the global performance of the suggested “lasso DIF” method to the logistic regression and Mantel-Haenszel methods, and to evaluate the different optimal penalty parameter selection methods. It is concluded that for small samples the lasso DIF approach globally outperforms the logistic regression method, and also the Mantel-Haenszel method, especially in the presence of item impact, while it yields similar results with larger samples. [less ▲]

Detailed reference viewed: 82 (11 ULg)
Full Text
See detailWhat’s beyond Concerto: an introduction to the R package catR
Magis, David ULg

Scientific conference (2014, June 10)

Detailed reference viewed: 26 (1 ULg)
Full Text
Peer Reviewed
See detailToward a Characterization of Western Operatic Singing Voices
Larrouy, Pauline ULg; Magis, David ULg; Nowak, Marion et al

Poster (2014, May 29)

Objective: One can easily recognize the musical style of a singer by listening to his/her sung performance. Several acoustical parameters of the Western operatic singing technique have been studied ... [more ▼]

Objective: One can easily recognize the musical style of a singer by listening to his/her sung performance. Several acoustical parameters of the Western operatic singing technique have been studied. However, the number of parameters could be extended and the effect of melody on these parameters remains unclear. By observing the effects of melody and technique on acoustical and musical parameters of the singing voice, this study aims at further characterizing the Western operatic singing technique. Methods: Fifty professional singers performed two contrasting melodies (popular song and romantic melody) with two vocal techniques (with and without operatic singing technique). The common quality parameters (energy distribution, vibrato rate and extent), perturbation parameters (standard deviation of the fundamental frequency, signal-to-noise ratio, jitter and shimmer) and musical features (fundamental frequency of the starting note, average tempo, and sound pressure level) of the 200 sung performances were analyzed. Results: The results show that the choice of melody had a limited impact on the acoustical and musical parameters observed, whereas a particular vocal profile appeared depending on the vocal technique employed. By examining these parameters in a theoretical model, this study highlights the relevance of vibrato rate, sound level, energy distribution, fundamental frequency of the starting note and tempo in describing the Western operatic singing technique. Conclusions: This study confirms that vocal technique affects most of the parameters examined and that the effect of melody is limited. In addition, the observation of quality and musical parameters contributes to a better understanding of the operatic singing technique. Conversely, the perturbation parameters don’t seem to take part in the characterization of operatic singing voices. Although the suggested theoretical model needs to be further developed in future research, it already generates implications for research and teaching. [less ▲]

Detailed reference viewed: 19 (0 ULg)
Full Text
See detailOpen-source CAT software: R packages and Concerto
Magis, David ULg

Conference (2014, March 12)

Together with the investigation of new or updated CAT procedures, it is of primary importance to ensure development of appropriate, flexible and useful CAT software. Open-source CAT algorithms have been ... [more ▼]

Together with the investigation of new or updated CAT procedures, it is of primary importance to ensure development of appropriate, flexible and useful CAT software. Open-source CAT algorithms have been recently proposed and are offering very promising tools for future practical CAT implementations, though yet under development. After a brief overview of available (commercial) software, I will present and compare the characteristics of some open-source R packages as CAT solutions: catR (Magis, & Raîche, 2012), catIrt (Nydick, 2013) and MAT (Choi, 2011), as well as the R-based software Firestar (Choi, 2009). A more complete description of catR will be given and (depending on time and computer constraints) a short illustrative session will be proposed. Finally, the web platform Concerto (Kosinski & Rust, 2011) will be shortly introduced. References: Choi, S. W. (2009). Firestar: Computerized adaptive testing simulation program for polytomous item response theory models. Applied Psychological Measurement, 33, 644-645. Choi, S. W. (2011). MAT: Multidimensional Adaptive Testing (MAT). R package version 0.1-3. Kosinski, M., & Rust, J. (2011). The development of Concerto: An open source online adaptive testing platform. Paper presented at the International Association for Computerized Adaptive Testing, Pacific Grove, CA. Magis, D., & Raîche, G. (2012). Random generation of response patterns under computerized adaptive testing with the R package catR. Journal of Statistical Software, 48 (8), 1-31. Nydick, S. W. (2013). catIrt: An R package for simulating IRT-based computerized adaptive tests. R package version 0.4-1. [less ▲]

Detailed reference viewed: 193 (2 ULg)
Full Text
See detailAn efficient standard error formula for the weighted likelihood estimator of ability
Magis, David ULg

Scientific conference (2014, March 04)

The weighted likelihood estimator (WLE; Warm, 1989) has become a very popular ability estimator in item response theory. It was developed to basically cancel the estimation bias of the maximum likelihood ... [more ▼]

The weighted likelihood estimator (WLE; Warm, 1989) has become a very popular ability estimator in item response theory. It was developed to basically cancel the estimation bias of the maximum likelihood estimator (MLE) with short tests. However, some uncertainty remains about its standard error formula. Warm (1989) established the asymptotic equivalence of the standard errors of both MLE and WLE, but there exists actually various practical formulas for the latter (Magis & Raîche, 2012; Nydick, 2013; Partchev, 2012; Warm, 2007), leading obviously to confusion. The purpose of this talk is to briefly sketch a general approach to derive a first-order approximation of the standard error of the WLE. Based on asymptotic adjustments of the law of large numbers and central limit theorem, a simple formula is derived. The efficiency of the latter is compared to aforementioned formulas by means of a simulation study under Rasch modeling. Preliminary results indicate that the derived formula has lower bias and RMSE than any other competing formula, especially with short tests. All formulas behave similarly with longer tests, as expected. [less ▲]

Detailed reference viewed: 20 (1 ULg)
Full Text
Peer Reviewed
See detailRecent advances and improvements in computerized adaptive testing with the R package catR
Magis, David ULg; Barrada, Juan

Conference (2014, February 14)

The purpose of this talk is to present recent advances and developments of the R package catR. This package allows for random generation of response patterns under computerized adaptive testing (CAT) and ... [more ▼]

The purpose of this talk is to present recent advances and developments of the R package catR. This package allows for random generation of response patterns under computerized adaptive testing (CAT) and holds various options to select the first items, estimate ability, select the next item, stop the test and return final results. Two main improvements were realized. First, several rules for next item selection were added, among others, Kullback- Leibler, progressive and proportional methods. Second, catR was limited to dichotomous IRT models so far. The most recent update allows now for several polytomous IRT models, such as partial and generalized partial credit model, graded and modified graded response models, rating scale model, and nominal response model. All improvements will be shortly presented, both from a theoretical point of view and in terms of practical implementation in catR. If possible, several illustrative examples will be displayed in a short live demonstration of catR. [less ▲]

Detailed reference viewed: 55 (3 ULg)
Peer Reviewed
See detailAnalyse du fonctionnement différentiel des items des versions papier et informatisée d’un test de classement en anglais, langue seconde, en présence de patrons de réponses inappropriés
Béland, Sébastien; Magis, David ULg; Raîche, Gilles

in Blais, Jean-Guy; Gilles, Jean-Luc; Tristan-Lopez (Eds.) Bienvenue au 21e siècle : Evaluation des apprentissages et technologies de l’information et de la communication (2014)

Detailed reference viewed: 19 (1 ULg)
Full Text
Peer Reviewed
See detailOn the asymptotic standard error of a class of robust estimators of ability in dichotomous item response models
Magis, David ULg

in British Journal of Mathematical & Statistical Psychology (2014), 67

In item response theory, the classical estimators of ability are highly sensitive to response disturbances and can return strongly biased estimates of the true underlying ability level. Robust methods ... [more ▼]

In item response theory, the classical estimators of ability are highly sensitive to response disturbances and can return strongly biased estimates of the true underlying ability level. Robust methods were introduced to lessen the impact of such aberrant responses onto the estimation process. The computation of asymptotic (i.e., large sample) standard errors (ASE) for these robust estimators, however, has not been fully considered yet. This paper focuses on a broad class of robust ability estimators, defined by an appropriate selection of the weight function and the residual measure, for which the ASE is derived from the theory of estimating equations. The maximum likelihood (ML) and the robust estimators, together with their estimated ASE, are then compared through a simulation study. It is concluded that both the estimators and their ASE perform similarly in absence of response disturbances, while the robust estimator and its estimated ASE are less biased and outperform their ML counterparts in presence of response disturbances with large impact on the item response process. [less ▲]

Detailed reference viewed: 22 (4 ULg)
Full Text
Peer Reviewed
See detailThe effect of melody and technique on the singing voice accuracy of trained singers
Larrouy, Pauline ULg; Magis, David ULg; Morsomme, Dominique ULg

in Logopedics, Phoniatrics, Vocology (2014), 39

A previous study highlighted the effect of vocal technique on the singing voice accuracy of trained singers (1). The intervals' precision between the notes of the tune was altered when the singers used ... [more ▼]

A previous study highlighted the effect of vocal technique on the singing voice accuracy of trained singers (1). The intervals' precision between the notes of the tune was altered when the singers used Western operatic singing technique. In order to better understand these results, we have recorded two different melodies sung with two different vocal techniques. A large panel of trained singers (N = 50) participated in the study. The analytical method described in the reference paper (1) has been applied. The results confirm the effect of vocal technique on the vocal accuracy of trained singers. In addition, these results provide an answer about the melodic effect and guide future work on the perception process of operatic voices. [less ▲]

Detailed reference viewed: 60 (24 ULg)