The effect of melody and technique on the singing voice accuracy of trained singers

in Logopedics, Phoniatrics, Vocology (in press)

A previous study highlighted the effect of vocal technique on the singing voice accuracy of trained singers (1). The intervals' precision between the notes of the tune was altered when the singers used ... [more ▼]

A previous study highlighted the effect of vocal technique on the singing voice accuracy of trained singers (1). The intervals' precision between the notes of the tune was altered when the singers used Western operatic singing technique. In order to better understand these results, we have recorded two different melodies sung with two different vocal techniques. A large panel of trained singers (N = 50) participated in the study. The analytical method described in the reference paper (1) has been applied. The results confirm the effect of vocal technique on the vocal accuracy of trained singers. In addition, these results provide an answer about the melodic effect and guide future work on the perception process of operatic voices. [less ▲]

Estimation des paramètres d’item et de sujet à partir du modèle de Rasch : une étude comparative des logiciels BILOG-MG, ICL et R

in Mesure et Evaluation en Education [=MEE] (in press)

La théorie de la réponse aux items (TRI) est une classe de modèles de mesure très utilisée en éducation. À ce jour, de nombreux logiciels, tel BILOG-MG, sont disponibles afin de procéder à l'estimation ... [more ▼]

La théorie de la réponse aux items (TRI) est une classe de modèles de mesure très utilisée en éducation. À ce jour, de nombreux logiciels, tel BILOG-MG, sont disponibles afin de procéder à l'estimation des paramètres d'item et de sujet. Parmi ces logiciels, il ne faut pas négliger ICL et R, qui sont gratuits et qui peuvent permettre de produire des analyses diversifiées. Cette étude a pour objectif de comparer la qualité d’estimation des paramètres selon une des modélisations issues de la TRI : le modèle de Rasch. Pour ce faire, nous comparons les estimateurs du paramètre de difficulté et de sujet selon trois logiciels : BILOG-MG, ICL et la librairie ltm disponible dans le logiciel R. Nous procédons à une analyse par simulation informatique et, dans un second temps, nous analysons un test de classement en anglais, langue seconde. Les résultats démontrent que les logiciels étudiés permettent d’obtenir des estimateurs des paramètres similaires, la différence principale entre ces logiciels étant leur temps d’exécution des procédures d’estimation. [less ▲]

A small overview of available computer software to support computerized adaptive testing

Conference (2013, August 27)

Computerized adaptive testing (CAT) is becoming a central tool for testing and assessment. It offers many advantages over fixed (“paper-and-pencil”) methods, such as individualized assessment, reduction ... [more ▼]

Computerized adaptive testing (CAT) is becoming a central tool for testing and assessment. It offers many advantages over fixed (“paper-and-pencil”) methods, such as individualized assessment, reduction of fraud, and straightforward estimation of proficiency levels. CAT has been studied for decades and remains an up-to-date research field in psychometrics and educational science. Practical CAT administration, however, is less frequently considered in such studies. Assigning CAT to respondents requires both the sufficient availability of computer machines, and the use of a powerful and easy-to-use CAT software. With the fast increase of computer resources at moderate cost, the availability of computer machines is becoming a less central, yet important, issue in the practical assessment of CAT tests. The choice of an accurate CAT software, on the other hand, should be guided by its flexibility, its underlying statistical modeling, and its user-friendly potential. According to the type of research or data analysis, some CAT software might be preferred to another. It is therefore important for the researcher or the clinician to know about the current availability of such software, in line with current research and practice in the CAT framework. Moreover, these software should allow enough flexibility to incorporate updates and new theoretical developments, such as e.g., new rules for next item selection. This talk proposes a simple and user-oriented presentation of several CAT software that are currently available. The software to be presented are: the Firestar software (Choi, 2009), the R package catR (Magis & Raîche, 2012), the R package catIrt (Nydick, 2012) and the CAT web-platform Concerto (Kosinski & Rust, 2011). The first three are non-commercial software, while Concerto is a web interface between end users (willing to develop computerized assessment tests) and catR (as underlying routine software). Both R packages are written to be most useful for researchers, without end-user interface, and are therefore less appealing for applied researchers who are not familiar with R. Yet, they offer flexible solutions by means of many options to optimize the design of the test and generate many response patterns for further analyses. Also, they can be easily integrated as sub-routines for more sophisticated CAT software. Firestar provides a user interface and makes all necessary computations with underlying R code. This talk aims at focusing on freely available CAT software. For this reason, only the four aforementioned programs will be presented, although it exists other, commercial CAT software such as e.g., the CATSim software (Assessment Systems Corporation, 2012). The different CAT software are briefly presented and their advantages and drawbacks, flexibility and usefulness are compared, mostly from the point of view of the applied researcher and clinician. The following criteria were retained for objective comparison: (a) their main goal of application; (b) the type of data and IRT modeling they can deal with; (c) the type of users they are focusing on; (d) their operating options; (e) their availability and flexibility for further improvements. A small demonstration of the R package catR will be proposed optionally, depending on time limitation. References Assessment Systems Corporation (2012). CATSim: Comprehensive simulation of computerized adaptive testing. St. Paul, MN. URL: http://www.assess.com/. Choi , S. W. (2009). Firestar: Computerized adaptive testing simulation program for polytomous item response theory models. Applied Psychological Measurement, 33, 644-645. Kosinski, M., & Rust, J. (2011). The development of Concerto: An open source online adaptive testing platform. Paper presented at the International Association for Computerized and Adaptive Testing (IACAT), Pacific Grove, CA. Magis, D., & Raîche, G. (2012). Random generation of response patterns under computerized adaptive testing with the R package catR. Journal of Statistical Software, 48, 1-31. Nydick, S. W. (2012). catIrt: An R package for simulating IRT-based computerized adaptive tests. R package version 0.3-0. [less ▲]

Proposition de nouveaux indices de détection de patrons de réponses inappropriés dans le contexte des enquêtes et des épreuves d’évaluation des apprentissages

Conference (2013, June 07)

Il n’est pas rare de voir des étudiants répondre de façon inappropriée à une épreuve d’évaluation comportant des items à réponses choisies. Par exemple, certains individus peuvent tricher alors que ... [more ▼]

Il n’est pas rare de voir des étudiants répondre de façon inappropriée à une épreuve d’évaluation comportant des items à réponses choisies. Par exemple, certains individus peuvent tricher alors que d’autres peuvent tenter de se sous-classer intentionnellement à un examen. Plusieurs approches ont été développées pour faire la détection de ce type d’étudiants. À ce jour, l’approche la plus prometteuse est l’utilisation d’indice de détection de type person‐fit (Meijer et Sijtsma, 2001). L’indice de détection lz (Drasgow, Levine et Williams, 1985) est fort probablement le plus utilisé et le plus connu d’entre tous. Malheureusement, cet indice est fortement affecté par le fait que l’habileté des étudiants est estimée et non pas réelle ; ce qui peut biaiser son calcul (Molenaar et Hoijtink, 1990). Pour pallier ce problème, Snijders (2001) a proposé une correction qui permet de diminuer considérablement le biais associé à la moyenne et à la variance de l’indice lz. Dans le cadre de notre projet doctoral, nous nous inspirerons de la suggestion de Snijders (2001) pour corriger deux autres indices de détection de patrons de réponses inappropriés : l’infit mean square (u) et l’outfit mean square (w). À cette fin, nous utiliserons une approche monte carliste afin d’investiguer plus en détails l’erreur de type 1 et la puissance de ces autres indices. Nos résultats préliminaires, que nous présenterons lors de cette communication, montrent que ces autres indices corrigés semblent eux aussi plus efficaces que leur version traditionnelle sans correction. [less ▲]

Application of lasso penalization to differential item functioning detection

Scientific conference (2013, February 26)

Identification of differential item functioning (DIF) in dichotomously scored items is often performed item by item. This approach increases the risk of false discovery errors (Type I error rate) as all ... [more ▼]

Identification of differential item functioning (DIF) in dichotomously scored items is often performed item by item. This approach increases the risk of false discovery errors (Type I error rate) as all items other than the tested one are assumed to be free of DIF. Some ad-hoc procedures, such as item purification and alpha level adjustment for multiple comparisons, have been studied in this context. The purpose of this talk is to focus on a different approach based on penalized likelihood estimation of a look-alike IRT model. Specifically, a Rasch model is being introduced with item-group interaction terms (i.e. DIF effects). Rather than obtaining pointwise estimates of the interaction parameters, which may be impossible because of high collinearity effects, the DIF effects are estimated with a lasso penalty term. Several criteria for optimally selecting the lasso tuning parameter are discussed, including cross-validation, AIC, BIC, and variants of these criteria. Preliminary results of a simulation study are presented and discussed. [less ▲]

An overview of statistical methods to assess differential item functioning and differential test functioning

Scientific conference (2013, February 12)

This talk broadly focuses on the identification of differential item functioning (DIF) and differential test functioning (DTF). After a short introduction of the key concepts, most-known methods to detect ... [more ▼]

This talk broadly focuses on the identification of differential item functioning (DIF) and differential test functioning (DTF). After a short introduction of the key concepts, most-known methods to detect DIF and DTF with dichotomously or polytomously scored items, and between two or more than two groups, are presented. Both parametric (i.e. IRT) and nonparametric (i.e. score-based) methods are described in a non-technical way. Several potential applications to PISA surveys are discussed. [less ▲]

Évaluation d’un test de lecture en anglais par deux méthodes de détection du fonctionnement différentiel d’items

in Revue des Sciences de l'Education (2013), 37

Cette étude vise à examiner la présence de fonctionnement différentiel d’items selon le sexe des répondants dans un test de compréhension en lecture en anglais administré à 171 universitaires francophones ... [more ▼]

Cette étude vise à examiner la présence de fonctionnement différentiel d’items selon le sexe des répondants dans un test de compréhension en lecture en anglais administré à 171 universitaires francophones. Deux méthodes non paramétriques sont utilisées: le test Mantel-Haenszel et le modèle de régression logistique. Sur un total de 64 items, deux présentent un fonctionnement différentiel selon le test Mantel-Haenszel, alors que cinq items supplémentaires ressortent par la régression logistique. Ce faible nombre d’items suggère une bonne équité du test, mais les différences observées soulignent la nécessité d’analyses additionnelles pour clarifier le statut de ces items. [less ▲]

Non-graphical solutions to Cattell's scree test

in Methodology: European Journal of Research Methods for the Behavioral and Social Sciences (2013), 9

Most of the strategies that have been proposed to determine the number of components that account for the most variation in a principal components analysis of a correlation matrix rely on the analysis of ... [more ▼]

Most of the strategies that have been proposed to determine the number of components that account for the most variation in a principal components analysis of a correlation matrix rely on the analysis of the eigenvalues and on numerical solutions. The Cattell’s scree test is a graphical strategy with a nonnumerical solution to determine the number of components to retain. Like Kaiser’s rule, this test is one of the most frequently used strategies for determining the number of components to retain. However, the graphical nature of the scree test does not definitively establish the number of components to retain. To circumvent this issue, some numerical solutions are proposed, one in the spirit of Cattell’s work and dealing with the scree part of the eigenvalues plot, and one focusing on the elbow part of this plot. A simulation study compares the efficiency of these solutions to those of other previously proposed methods. Extensions to factor analysis are possible and may be particularly useful with many low-dimensional components. [less ▲]

Duration perception: A Developmental Semiology from 2:6 to 13 Years, General Population versus Atypical, Assessment by Parental Questionnaire

Conference (2012, December 20)

Introduction Perception of duration has a major impact on the child’s ability to manage everyday life. Usually, the perception of time is considered in its sequential component. To our knowledge, possible ... [more ▼]

Introduction Perception of duration has a major impact on the child’s ability to manage everyday life. Usually, the perception of time is considered in its sequential component. To our knowledge, possible semiological markers for the capacity to apprehend temporal duration have not yet been explored. There are thus no existing tools either for its evaluation (in everyday life) or for the description of its development during childhood. We are thus largely in the dark about how the perception of duration develops during childhood. Hypothesis 1) It is possible to track perception of duration using semiological markers in everyday life; 2) Children with developmental disorders can be expected to develop difficulties in this domain. Method We designed a questionnaire for parents using a Likert scale. The questionnaire was administered to 2 samples from 2:6 to 13 years: 827 in a general population and 297 in a mixed-psychopathological population recruited from the Mental Health Services network. We carried out a multinomial logistic regression and used percentiles curves (P5,50,95). Results In the general population, perception of duration is gradually acquired and full competence is attained by 9 years. The apprehension of sequential time is achieved by 4 years. In the psychopathological sample, mean perception of temporal duration is delayed by 18 months and significant difficulties remain even at 13 years. The apprehension of sequential time is also delayed: even at 13 years it remains low (at percentile 95). We have found no sex-related differences. Discussion Perception of duration is an important developmental feature which has not been taken into account until now. Semiological markers can be used to differentiate typically developing children from clinical populations as to their respective capacities to perceive temporal duration. [less ▲]

A framework and approaches to develop an in-house CAT with freeware and open sources

Conference (2012, August 13)