The contribution of Multidimensional IRT for modeling applicants performance on a SJT.
Lothe, Benoit ; Monseur, Christian ; Hansez, Isabelle
Poster (2013, May)
Situational judgment tests (SJTs) are testing methods often used in Personnel Selection to appraise different constructs (Whetzel & McDaniel, 2009). In the framework of an overall career progression ... [more ▼]
Situational judgment tests (SJTs) are testing methods often used in Personnel Selection to appraise different constructs (Whetzel & McDaniel, 2009). In the framework of an overall career progression program in a Public Administration, SJTs were developed to assess several competencies relevant to both task (e.g. planning, managing, etc.) and contextual performance (e.g. helping behaviour, organizational loyalty, etc.). Each competency is a combination of KSAOs needed to perform efficiently in a specific context (Campion & al., 2012; Le Boterf 2008). Given the lack of internal consistency of SJT and the dichotomous nature of the data, we decide to use Multidimensional Item Response Theory (MIRT) model to examine the issue of construct validity. Based on Hartig & Höhler (2009), we use MIRT compensatory model to estimate the latent covariance structures of the SJT. Different model are tested and compared depending on the number of dimension involved, the nature of the items and latent variables relation (between or within-item multidimensionality) and the number of model parameter (one or two parameter model). The results and findings lead us to discuss some implications (gaining accuracy, modeling performance in complex task, etc.) and limitations (dealing with complex loading structure, etc.) of using IRT, over and above classical test theory, in the specific case of SJTs applicants performance. The originality of this communication also lies in the way we addressed SJTs reliability and construct validity issues with a measurement based model that differs from the one usually used in the field. [less ▲]Detailed reference viewed: 79 (5 ULg)
Elaboration et validation de tests de jugement situationnel comme outil de sélection professionnelle : guide méthodologique
Lothe, Benoit ; Bertrand, Françoise ; Hansez, Isabelle
in Psychologie du Travail et des Organisations (2012), 18(3), 215-231Detailed reference viewed: 219 (10 ULg)
Development and validation of situational judgment tests for evaluating Organizational Citizenship Behavior and Contextual Performance Behavior in a professional context : methodological issues.
Lothe, Benoit ; Hansez, Isabelle
Conference (2011, June 22)
Strategic Human Resource Management typically raises appraisal issues (Analoui, 2007; Jackson, Schuler & Werner, 2009 ; Teo & Crawford, 2005) mostly related to the HRM relationship with organizational ... [more ▼]
Strategic Human Resource Management typically raises appraisal issues (Analoui, 2007; Jackson, Schuler & Werner, 2009 ; Teo & Crawford, 2005) mostly related to the HRM relationship with organizational performance (Bowen & Ostroff, 2004 ; Cascio, 1996 ; Wright, Gardner, Moynihan & Allen, 2005). Hence workforce assessment remains a key success factor for many organizations. Thus this paper takes place in the early beginning of a PhD Thesis which aims to the development and validation of an assessment test of Organizational Citizenship Behavior (OCB) and Contextual Performance (CP) in a professional selection context. The managerial concept of OCB refers to “individual behavior that is discretionary, not directly or explicitly recognized by the formal reward system, and in the aggregate promotes the efficient and effective functioning of the organization” (Organ, Podsakoff & MacKenzie, 2006, p. 3). CP is a closely related construct, distinct of task performance, that encompasses behaviors supporting the psychological and social context in which task activities are performed (Stone-Romero, Alvarez, & Thompson, 2009). If these two frameworks look much alike, they contrast on several points: conceptual dimensions, discretional characteristics of behavior, etc. Therefore, it seems interesting to apprehend both commonly. Situational Judgment Tests (SJTs) show a good validity concerning prediction of job performance in employment settings (Lievens, Peeters, Schollaert, 2008 ; Whetzel & McDaniel, 2009). Furthermore, they are a multidimentional measurement method suited to assess different construct like job knowlege and skills, applied social skills, and some basic personality tendencies (Christian, Edwards, & Bradley, 2010). Taking into account its characteristics, SJTs seem to be an appropriate method to enable OCB & CP evaluation. This communication pursues three goals. The first is a theoretical issue: we need a straightforward vision of OCB & CP concepts. Accordingly, after a consistent review on the vast amount of work that have been written on these two concepts, we have to specify what we seek to appraise and make the inventory of existing measurement tools related to these focus. We have already worked out a structured methodology to develop and validate SJTs in selection and career management context (Lothe, Hansez & Bertrand, Submitted). So the second issue is a methodological one. How to improve and/or to adapt this methodology in order to create a new SJT measuring OCB & CP ? Thirdly, some research related issues conclude this paper. Through this PhD project, we should be able to examine several main issues that will shape the thesis. Are SJTs being able to provide a valid assessment of OCB & CP in a personnel selection context ? Does the discretionary nature of one behavior impact the quality of its measurement ? What is the effect of response instructions on OCB/CP situational judgment test performance ? Each issue will be illustrated and discussed in connection with the development and validation opportunities and constraints. [less ▲]Detailed reference viewed: 42 (8 ULg)
Competency Model validation process : an illustration in public administration sector
Lothe, Benoit ; Hansez, Isabelle ; Bertrand, Françoise
Poster (2011, May 26)
Background: Nowadays Human Resource Management issues are highly related to the strategic approach (Analoui, 2007; Daft, 2008; Becker, Huselid & Beatty, 2009 ; Jackson, Schuler & Werner, 2009). Workforce ... [more ▼]
Background: Nowadays Human Resource Management issues are highly related to the strategic approach (Analoui, 2007; Daft, 2008; Becker, Huselid & Beatty, 2009 ; Jackson, Schuler & Werner, 2009). Workforce is considered as an asset and HRM is required to match human capital with organization strategic purposes. HRM practices have accordingly evolved to better fulfil this requirement through for instance Competency Modelling (CM) (Werbel & De Marie, 2005). Diagnosis: This communication illustrates a specific CM validation process in a wide Public Administration. Through a “one-size-fits-all” competency model approach (Mansfield, 1996), the HRM department of this Public Administration has developed a set of 29 transverse competencies broken down across 290 behavioural indicators. After several in-house pre-validation steps (workshops and test cases), Human Resources Development Unit of University of Liege was invited to collaborate to the final validation process. Solution: Based on Van Beirendonck (2006) CM validation criteria, a two steps validation method was jointly developed. Four main criteria were used, therefore each behavioural indicator ought to : (1) meet various organizational departments needs; (2) be clearly understood within the organization (Bowen & Ostroff, 2004); (3) have a straight link with its belonging competency and (4) be observable. A questionnaire survey (n=252) was drawn up to appraise the first three criteria and quantitative analysis results provided the CM improvement framework. The concluding validation stage assessed the three last criteria by measuring Subject Matter Experts (HR managers responsible for staff evaluation) interrater agreement in order to improve CM content validity (Sanchez & Levine, 2009). Finding: Results comfortably confirm CM content validation under the four criteria. One of the project’s more relevant elements is developing a scientifically based method adapted to wide administration organizational features and constraints. In contrast, this CM validation project presents some limitations in relation to the CM theoretical concept (ambiguous concept in its validation perspective) and to organizational constraints (lack of strategic project concerning the use of the CM). Method and outcomes will be discussed in connection with the customer’s requirements and the constraints from the organizational context. [less ▲]Detailed reference viewed: 99 (3 ULg)
Conception et validation d'épreuves basées sur les compétences professionnelles : le cas du concours pour kinésithérapeutes indépendants.
; Lothe, Benoit ; Hansez, Isabelle
Conference (2010, July 09)Detailed reference viewed: 15 (3 ULg)
Comment valoriser un processus de sélection - évaluation professionnelle? Comparaison critique des méthodes d'utilité économique et d'évaluation financière.
Lothe, Benoit ; Hansez, Isabelle
Conference (2010, July 09)Detailed reference viewed: 25 (7 ULg)
Garantir aux fondements du caractère scientifique d'un dictionnaire des compétences : méthode croisée de validation et approche participative.
Ben Lamine, Boutheina ; Lothe, Benoit ; et al
Conference (2010, July 09)Detailed reference viewed: 56 (5 ULg)
Situational Judgment Tests : Illustration od Item analysis by Classical Test Theory and Item Response Theory Approaches.
Lothe, Benoit ; Bertrand, Françoise ; Hansez, Isabelle
Conference (2009, May 16)Detailed reference viewed: 46 (4 ULg)
Elaboration d'une épreuve d'accession à un niveau supérieur dans le secteur public : processus itératif de validation.
Lothe, Benoit ; ; Bertrand, Françoise et al
Conference (2008, August 22)Detailed reference viewed: 20 (5 ULg)