References of "Magis, David"
     in
Bookmark and Share    
Full Text
Peer Reviewed
See detailDetection of differential item functioning using the lasso approach
Magis, David ULg; Tuerlinckx, Francis; De Boeck, Paul

in Journal of Educational & Behavioral Statistics (in press)

This paper proposes a novel approach to detect differential item functioning (DIF) among dichotomously scored items. Unlike standard DIF methods that perform an item-by-item analysis, we propose the “LR ... [more ▼]

This paper proposes a novel approach to detect differential item functioning (DIF) among dichotomously scored items. Unlike standard DIF methods that perform an item-by-item analysis, we propose the “LR lasso DIF method”: logistic regression (LR) model is formulated for all item responses. The model contains item specific intercepts, an effect of the sum score and item-group interaction (i.e. DIF) effects, with a lasso penalty on all DIF parameters. Optimal penalty parameter selection is investigated through several known information criteria (AIC, BIC and cross-validation) as well as through a newly developed alternative. A simulation study was conducted to compare the global performance of the suggested “LR lasso DIF” method to the logistic regression and Mantel-Haenszel methods (in terms of false alarm and hit rates). It is concluded that for small samples the LR lasso DIF approach globally outperforms the logistic regression method, and also the Mantel-Haenszel method, especially in the presence of item impact, while it yields similar results with larger samples. [less ▲]

Detailed reference viewed: 28 (3 ULg)
Full Text
Peer Reviewed
See detailA note on the equivalence between observed and expected information functions with polytomous IRT models
Magis, David ULg

in Journal of Educational & Behavioral Statistics (in press)

The purpose of this note is to study the equivalence of observed and expected (Fisher) information functions with polytomous item response theory (IRT) models. It is established that observed and expected ... [more ▼]

The purpose of this note is to study the equivalence of observed and expected (Fisher) information functions with polytomous item response theory (IRT) models. It is established that observed and expected information functions are equivalent for the class of divide-by-total models (including partial credit, generalized partial credit, rating scale and nominal response models), but not for the class of difference models (including the graded response and modified graded response models). Yet, observed information function remains positive in both classes. Straightforward connections with dichotomous IRT models and further implications are outlined. [less ▲]

Detailed reference viewed: 12 (2 ULg)
Full Text
Peer Reviewed
See detailOn the asymptotic standard error of a class of robust estimators of ability in dichotomous item response models
Magis, David ULg

in British Journal of Mathematical & Statistical Psychology (in press)

In item response theory, the classical estimators of ability are highly sensitive to response disturbances and can return strongly biased estimates of the true underlying ability level. Robust methods ... [more ▼]

In item response theory, the classical estimators of ability are highly sensitive to response disturbances and can return strongly biased estimates of the true underlying ability level. Robust methods were introduced to lessen the impact of such aberrant responses onto the estimation process. The computation of asymptotic (i.e., large sample) standard errors (ASE) for these robust estimators, however, has not been fully considered yet. This paper focuses on a broad class of robust ability estimators, defined by an appropriate selection of the weight function and the residual measure, for which the ASE is derived from the theory of estimating equations. The maximum likelihood (ML) and the robust estimators, together with their estimated ASE, are then compared through a simulation study. It is concluded that both the estimators and their ASE perform similarly in absence of response disturbances, while the robust estimator and its estimated ASE are less biased and outperform their ML counterparts in presence of response disturbances with large impact on the item response process. [less ▲]

Detailed reference viewed: 21 (4 ULg)
Full Text
Peer Reviewed
See detailA note on weighted likelihood and Jeffreys modal estimation of proficiency levels in polytomous item response models
Magis, David ULg

in Psychometrika (in press)

Warm (1989) established the equivalence between the so-called Jeffreys modal and the weighted likelihood estimators of proficiency level with some dichotomous item response models. The purpose of this ... [more ▼]

Warm (1989) established the equivalence between the so-called Jeffreys modal and the weighted likelihood estimators of proficiency level with some dichotomous item response models. The purpose of this note is to extend this result to polytomous item response models. First, a general condition is derived to ensure the perfect equivalence between these two estimators. Second, it is shown that this condition is fulfilled by two broad classes of polytomous models including, among others, the partial credit, rating scale, graded response and nominal response models. [less ▲]

Detailed reference viewed: 28 (5 ULg)
Full Text
Peer Reviewed
See detailPassage de l’administration fixe d’un test à une administration adaptative : application au TCALS-II
Magis, David ULg; Raîche, Gilles

in Raîche, Gilles; Ndinga, Pascal; Meunier, Hélène (Eds.) L'interdisciplinarité de la mesure et de l’évaluation (in press)

La problématique du passage de l’administration fixe (de type papier-crayon) à une administration adaptative d’un test est étudiée. Une méthode en deux étapes est présentée. Dans un premier temps, des ... [more ▼]

La problématique du passage de l’administration fixe (de type papier-crayon) à une administration adaptative d’un test est étudiée. Une méthode en deux étapes est présentée. Dans un premier temps, des patrons de réponses sont générés selon une administration fixe, dans le but de déterminer des valeurs admissibles de l’erreur-type d’estimation du niveau d’habileté. Ensuite, ces valeurs sont utilisées comme critères d’arrêt lors d’une administration adaptative du même test. La longueur du test est alors considérée pour évaluer la qualité du test par rapport à sa version fixe. Le test de classement en anglais, langue seconde, au collégial (TCALS-II) est utilisé en guise d’illustration. Il est établi qu’une administration adaptative du TCALS-II permettrait de réduire sensiblement la longueur du test, sans perte de qualité de l’estimation des niveaux d’habileté. Toutefois, cette amélioration est limitée aux sujets ne présentant pas un niveau d’habileté trop faible ou trop important. [less ▲]

Detailed reference viewed: 23 (5 ULg)
Peer Reviewed
See detailLe testing adaptatif informatisé : une brève introduction
Magis, David ULg

Conference (2015, January 27)

L’objet de cet exposé est de présenter les grands principes et concepts du testing adaptatif informatisé (TAI). Les éléments abordés sont : le TAI face au test fixe (papier-crayon), les principes généraux ... [more ▼]

L’objet de cet exposé est de présenter les grands principes et concepts du testing adaptatif informatisé (TAI). Les éléments abordés sont : le TAI face au test fixe (papier-crayon), les principes généraux du TAI (banque d’items, estimation provisoire et finale de la compétence, sélection des items à administrer, règles d’arrêt), les principes spécifiques au TAI (contrôle de l’exposition des items, équilibrage du contenu des tests). L’exposé se veut didactique et général afin de dessiner les contours du TAI. Il se termine par un état de l’art sur les recherches actuelles sur le TAI. [less ▲]

Detailed reference viewed: 9 (1 ULg)
See detailPsychometrics and bibliometrics: Overview of future possible interactions
Magis, David ULg

Scientific conference (2014, December 02)

Bibliometrics is a growing research field with the main aim of collecting bibliometric information (citations, publications, awards...) to infer valuable rankings or to predict future academic performance ... [more ▼]

Bibliometrics is a growing research field with the main aim of collecting bibliometric information (citations, publications, awards...) to infer valuable rankings or to predict future academic performance. However, most bibliometric approaches are based either on pairwise comparisons of "objects" (authors, funding applications, academics...) with respect to selected "criteria" (publications, citation rate, h-index...), or on modeling observable academic outcomes using observed such criteria. The purpose of this talk is twofold. First, an improvement of the pairwise comparison approach will be outlined on the basis of so-called multicriteria decision aid (MCDA) routines, which will imply an increased flexibility and added freedom for the decision process. Second, we will start a deeper reflexion on possible connections between bibliometrics and psychometrics. Among others, the possibility of modeling latent "academic ability" as outcome of interest, given manifest criteria, will be put to an open discussion and exchange with the listeners [less ▲]

Detailed reference viewed: 8 (0 ULg)
Full Text
Peer Reviewed
See detailThe sentence repetition task: A powerful diagnostic tool for French children with specific language impairment
Leclercq, Anne-Lise ULg; Quémart, Pauline; Magis, David ULg et al

in Research in Developmental Disabilities (2014), 35

This study assesses the diagnostic accuracy and construct validity of a sentence repetition task that is commonly used for the identification of French children with specific language impairment (SLI ... [more ▼]

This study assesses the diagnostic accuracy and construct validity of a sentence repetition task that is commonly used for the identification of French children with specific language impairment (SLI). Thirty-four school-aged children with a confirmed, diagnostically based diagnosis of SLI, and 34 control children matched on age and nonverbal abilities performed the sentence repetition task. Two general scoring measures took into account the verbatim repetition of the sentence and the number of words accurately repeated. Moreover, five other scoring measures were applied to their answers in order to separately take into account their respect of lexical items, functional items, syntax, verb morphology, and the general meaning of the sentence. Results show good to high levels of sensitivity and specificity at the three cut-off points for all scoring measures. A principal component analysis revealed two factors. Scoring measures for the respect of functional words, syntax and verb morphology provided the largest loadings to the first factor, while scoring measures for the respect of lexical words and general semantics provided the largest loadings to the second factor. Sentence repetition appears to be a valuable tool to identify SLI in French children, and the ability to repeat sentences correctly is supported by two factors: a morphosyntactic factor and a lexical factor. [less ▲]

Detailed reference viewed: 40 (9 ULg)
See detailA lasso penalization approach to differential item functioning
Magis, David ULg

Conference (2014, November 24)

Detailed reference viewed: 11 (2 ULg)
Full Text
Peer Reviewed
See detailOn the finiteness and uniqueness of the weighted likelihood estimator of ability in polytomous IRT models
Magis, David ULg; Verhelst, Norman

Conference (2014, November 21)

In item response theory (IRT), the weighted likelihood (WL) estimator has become a central method to estimate ability levels of respondents. Primarily introduced with dichotomous IRT models (Warm, 1989 ... [more ▼]

In item response theory (IRT), the weighted likelihood (WL) estimator has become a central method to estimate ability levels of respondents. Primarily introduced with dichotomous IRT models (Warm, 1989), it was later extended to polytomous IRT models (Samejima, 1998). However, very few information is available about the behavior of the WL estimator, and especially about the uniqueness of the ability estimates as well as their finiteness. The purpose of this talk is to establish that with polytomous item response models, the WL estimator of ability always returns finite values. This result is valid for the class of difference models and divide-by-total models, independently of the number of items and the response patterns. However, such estimates may not necessarily be unique, as the WL solving equation may provide several optimal values. Some examples are considered to illustrate both findings. [less ▲]

Detailed reference viewed: 24 (3 ULg)
Peer Reviewed
See detailRecent advances and improvements in computerized adaptive testing with the R package catR
Barrada, Juan Ramon; Magis, David ULg

Conference (2014, October 08)

The purpose of this talk is to present recent advances and developments of the R package catR. This package allows for random generation of response patterns under computerized adaptive testing (CAT) and ... [more ▼]

The purpose of this talk is to present recent advances and developments of the R package catR. This package allows for random generation of response patterns under computerized adaptive testing (CAT) and holds various options to select the first items, estimate ability, select the next item, stop the test and return final results. Two main improvements were realized. First, several rules for next item selection were added, among others, Kullback- Leibler, progressive and proportional methods. Second, catR was limited to dichotomous IRT models so far. The most recent update allows now for several polytomous IRT models, such as partial and generalized partial credit model, graded and modified graded response models, rating scale model, and nominal response model. All improvements will be shortly presented, both from a theoretical point of view and in terms of practical implementation in catR. If possible, several illustrative examples will be displayed in a short live demonstration of catR. [less ▲]

Detailed reference viewed: 45 (0 ULg)
Full Text
Peer Reviewed
See detailDetection of differential item functioning using the lasso approach
Magis, David ULg; Tuerlinckx, Francis; De Boeck, Paul

Conference (2014, July 22)

The purpose of this talk is to present a novel approach to detect differential item functioning (DIF) among dichotomously scored items. Unlike standard DIF methods that perform an item-by-item analysis ... [more ▼]

The purpose of this talk is to present a novel approach to detect differential item functioning (DIF) among dichotomously scored items. Unlike standard DIF methods that perform an item-by-item analysis, we consider a logistic regression model including item-group interaction (i.e. DIF) effects of all items simultaneously. The method is based on penalized maximum likelihood estimation of a model with a lasso penalty on all possible DIF parameters. Optimal penalty parameter selection is investigated through several known information criteria (such as AIC and BIC) as well as a newly developed weighted alternative. A simulation study was conducted to compare the global performance of the suggested “lasso DIF” method to the logistic regression and Mantel-Haenszel methods, and to evaluate the different optimal penalty parameter selection methods. It is concluded that for small samples the lasso DIF approach globally outperforms the logistic regression method, and also the Mantel-Haenszel method, especially in the presence of item impact, while it yields similar results with larger samples. [less ▲]

Detailed reference viewed: 78 (11 ULg)
Full Text
See detailWhat’s beyond Concerto: an introduction to the R package catR
Magis, David ULg

Scientific conference (2014, June 10)

Detailed reference viewed: 26 (1 ULg)
Full Text
Peer Reviewed
See detailToward a Characterization of Western Operatic Singing Voices
Larrouy, Pauline ULg; Magis, David ULg; Nowak, Marion et al

Poster (2014, May 29)

Objective: One can easily recognize the musical style of a singer by listening to his/her sung performance. Several acoustical parameters of the Western operatic singing technique have been studied ... [more ▼]

Objective: One can easily recognize the musical style of a singer by listening to his/her sung performance. Several acoustical parameters of the Western operatic singing technique have been studied. However, the number of parameters could be extended and the effect of melody on these parameters remains unclear. By observing the effects of melody and technique on acoustical and musical parameters of the singing voice, this study aims at further characterizing the Western operatic singing technique. Methods: Fifty professional singers performed two contrasting melodies (popular song and romantic melody) with two vocal techniques (with and without operatic singing technique). The common quality parameters (energy distribution, vibrato rate and extent), perturbation parameters (standard deviation of the fundamental frequency, signal-to-noise ratio, jitter and shimmer) and musical features (fundamental frequency of the starting note, average tempo, and sound pressure level) of the 200 sung performances were analyzed. Results: The results show that the choice of melody had a limited impact on the acoustical and musical parameters observed, whereas a particular vocal profile appeared depending on the vocal technique employed. By examining these parameters in a theoretical model, this study highlights the relevance of vibrato rate, sound level, energy distribution, fundamental frequency of the starting note and tempo in describing the Western operatic singing technique. Conclusions: This study confirms that vocal technique affects most of the parameters examined and that the effect of melody is limited. In addition, the observation of quality and musical parameters contributes to a better understanding of the operatic singing technique. Conversely, the perturbation parameters don’t seem to take part in the characterization of operatic singing voices. Although the suggested theoretical model needs to be further developed in future research, it already generates implications for research and teaching. [less ▲]

Detailed reference viewed: 18 (0 ULg)
Full Text
See detailOpen-source CAT software: R packages and Concerto
Magis, David ULg

Conference (2014, March 12)

Together with the investigation of new or updated CAT procedures, it is of primary importance to ensure development of appropriate, flexible and useful CAT software. Open-source CAT algorithms have been ... [more ▼]

Together with the investigation of new or updated CAT procedures, it is of primary importance to ensure development of appropriate, flexible and useful CAT software. Open-source CAT algorithms have been recently proposed and are offering very promising tools for future practical CAT implementations, though yet under development. After a brief overview of available (commercial) software, I will present and compare the characteristics of some open-source R packages as CAT solutions: catR (Magis, & Raîche, 2012), catIrt (Nydick, 2013) and MAT (Choi, 2011), as well as the R-based software Firestar (Choi, 2009). A more complete description of catR will be given and (depending on time and computer constraints) a short illustrative session will be proposed. Finally, the web platform Concerto (Kosinski & Rust, 2011) will be shortly introduced. References: Choi, S. W. (2009). Firestar: Computerized adaptive testing simulation program for polytomous item response theory models. Applied Psychological Measurement, 33, 644-645. Choi, S. W. (2011). MAT: Multidimensional Adaptive Testing (MAT). R package version 0.1-3. Kosinski, M., & Rust, J. (2011). The development of Concerto: An open source online adaptive testing platform. Paper presented at the International Association for Computerized Adaptive Testing, Pacific Grove, CA. Magis, D., & Raîche, G. (2012). Random generation of response patterns under computerized adaptive testing with the R package catR. Journal of Statistical Software, 48 (8), 1-31. Nydick, S. W. (2013). catIrt: An R package for simulating IRT-based computerized adaptive tests. R package version 0.4-1. [less ▲]

Detailed reference viewed: 191 (2 ULg)
Full Text
See detailAn efficient standard error formula for the weighted likelihood estimator of ability
Magis, David ULg

Scientific conference (2014, March 04)

The weighted likelihood estimator (WLE; Warm, 1989) has become a very popular ability estimator in item response theory. It was developed to basically cancel the estimation bias of the maximum likelihood ... [more ▼]

The weighted likelihood estimator (WLE; Warm, 1989) has become a very popular ability estimator in item response theory. It was developed to basically cancel the estimation bias of the maximum likelihood estimator (MLE) with short tests. However, some uncertainty remains about its standard error formula. Warm (1989) established the asymptotic equivalence of the standard errors of both MLE and WLE, but there exists actually various practical formulas for the latter (Magis & Raîche, 2012; Nydick, 2013; Partchev, 2012; Warm, 2007), leading obviously to confusion. The purpose of this talk is to briefly sketch a general approach to derive a first-order approximation of the standard error of the WLE. Based on asymptotic adjustments of the law of large numbers and central limit theorem, a simple formula is derived. The efficiency of the latter is compared to aforementioned formulas by means of a simulation study under Rasch modeling. Preliminary results indicate that the derived formula has lower bias and RMSE than any other competing formula, especially with short tests. All formulas behave similarly with longer tests, as expected. [less ▲]

Detailed reference viewed: 19 (1 ULg)
Full Text
Peer Reviewed
See detailRecent advances and improvements in computerized adaptive testing with the R package catR
Magis, David ULg; Barrada, Juan

Conference (2014, February 14)

The purpose of this talk is to present recent advances and developments of the R package catR. This package allows for random generation of response patterns under computerized adaptive testing (CAT) and ... [more ▼]

The purpose of this talk is to present recent advances and developments of the R package catR. This package allows for random generation of response patterns under computerized adaptive testing (CAT) and holds various options to select the first items, estimate ability, select the next item, stop the test and return final results. Two main improvements were realized. First, several rules for next item selection were added, among others, Kullback- Leibler, progressive and proportional methods. Second, catR was limited to dichotomous IRT models so far. The most recent update allows now for several polytomous IRT models, such as partial and generalized partial credit model, graded and modified graded response models, rating scale model, and nominal response model. All improvements will be shortly presented, both from a theoretical point of view and in terms of practical implementation in catR. If possible, several illustrative examples will be displayed in a short live demonstration of catR. [less ▲]

Detailed reference viewed: 54 (3 ULg)
Full Text
Peer Reviewed
See detailThe effect of melody and technique on the singing voice accuracy of trained singers
Larrouy, Pauline ULg; Magis, David ULg; Morsomme, Dominique ULg

in Logopedics, Phoniatrics, Vocology (2014), 39

A previous study highlighted the effect of vocal technique on the singing voice accuracy of trained singers (1). The intervals' precision between the notes of the tune was altered when the singers used ... [more ▼]

A previous study highlighted the effect of vocal technique on the singing voice accuracy of trained singers (1). The intervals' precision between the notes of the tune was altered when the singers used Western operatic singing technique. In order to better understand these results, we have recorded two different melodies sung with two different vocal techniques. A large panel of trained singers (N = 50) participated in the study. The analytical method described in the reference paper (1) has been applied. The results confirm the effect of vocal technique on the vocal accuracy of trained singers. In addition, these results provide an answer about the melodic effect and guide future work on the perception process of operatic voices. [less ▲]

Detailed reference viewed: 59 (23 ULg)
Full Text
Peer Reviewed
See detailThe evaluation of vocal pitch accuracy: The case of operatic singing voices
Larrouy, Pauline ULg; Magis, David ULg; Morsomme, Dominique ULg

in Music Perception (2014), 32(1), 1-10

The objective analysis of Western operatic singing voices indicates that professional singers can be particularly “out of tune”. This study aims to better understand the evaluation of operatic voices ... [more ▼]

The objective analysis of Western operatic singing voices indicates that professional singers can be particularly “out of tune”. This study aims to better understand the evaluation of operatic voices, which have particularly complex acoustical signals. Twenty-two music experts were asked to evaluate the vocal pitch accuracy of 14 sung performances with a pairwise comparison paradigm, in a test and a retest. In addition to the objective measurement of pitch accuracy (pitch interval deviation), several performance parameters (average tempo, fundamental frequency of the starting note) and quality parameters (energy distribution, vibrato rate and extent) were observed and compared to the judges’ perceptual rating. The results show high intra- and inter-judge reliability when rating the pitch accuracy of operatic singing voices. Surprisingly, all the parameters were significantly related to the ratings and explain 78.8% of the variability of the judges’ rating. The pitch accuracy evaluation of operatic voices is thus not based exclusively on the precision of performed music intervals but on a complex combination of performance and quality parameters. [less ▲]

Detailed reference viewed: 38 (14 ULg)
Full Text
Peer Reviewed
See detaildeltaPlotR: An R package for differential item functioning analysis with Angoff's Delta Plot
Magis, David ULg; Facon, Bruno

in Journal of Statistical Software (2014), 59(1), 1-19

Angoff's delta plot is a straightforward and not computationally intensive method to identify differential item functioning (DIF) among dichotomously scored items. This approach was recently improved by ... [more ▼]

Angoff's delta plot is a straightforward and not computationally intensive method to identify differential item functioning (DIF) among dichotomously scored items. This approach was recently improved by proposing an optimal threshold selection and by considering several item purification processes. Moreover, to support practical DIF analyses with the delta plot and these improvements, the R package deltaPlotR was also developed. The purpose of this paper is twofold: to outline the delta plot by describing the original method and its recent improvements in a user-oriented way, and to illustrate the structure and performances of the deltaPlotR package. A real data set about language skill assessment is being analyzed as an illustrative example. [less ▲]

Detailed reference viewed: 18 (1 ULg)