PSYSCIQ 2009-1

ABSTRACTS

Quality of the Critical Incident Technique in practice: Interrater reliability and users’ acceptance under real conditions
ANNA KOCH, ANJA STROBEL, GÜLER KICI & KARL WESTHOFF

Abstract
The Critical Incident Technique (CIT) is a widely used task analysis method in personnel psychology. While studies on psychometric properties of the CIT so far primarily took into account relevance ratings of task-lists or attributes, and hence, only a smaller or adapted part of the CIT, little is known about the psychometric properties of the complete CIT in its most meaningful and fruitful way. Therefore, the aim of the present study was to assess interrater reliability and the participants` view of the CIT under real conditions and especially to provide data for the key step of the CIT: the classification of behavior descriptions into requirements. Additionally, the cost-benefit-ratio and practicability were rated from the participants’ views as an important indicator for the acceptance of the task analysis approach in practice. Instructors of German Institutions for Statutory Accidents Insurance and Prevention as well as their supervisors took part in a job analysis with the CIT. Moderate interrater reliability for the relevance rating was found while the classification step yielded unexpectedly low coefficients for interrater reliability. The cost-benefit-ratio and practicability of the complete CIT were rated very positive. The results are discussed in relation to determinants that facilitate or impede the application of task analysis procedures.

Keywords: critical incident technique, interrater reliability, acceptance, practical aspects, task analysis

Anna Koch, M.Sc.
Dresden University of Technology
Department of Psychology
Assessment and Intervention
01062 Dresden, Germany
E-Mail: anna.koch@tu-dresden.de

Feminization of management leads to backlash against agentic applicants: Lack of social skills, not gender, determines low hireability judgments in a student sample
MELANIE C. STEFFENS, JANETTE C. SCHULT & IRENA D. EBERT

Abstract
Effects of an applicant`s sex on the ascription of task competence, social skills, and hireability are of theoretical as well as practical concern. In two influential series of studies with American college students, Rudman and Glick (1999, 2001) showed a backlash effect: When social skills were included in job descriptions (“feminized job descriptions”), female, but not male applicants displaying agentic behaviour were rated low in social skills and considered less hireable than their male counterparts. We tested the generality of this backlash effect in a replication with German students. In four experiments a total of 555 participants listened to telephone interviews with alleged applicants. Female and male applicants were rated very similarly with respect to task competence, social skills, and hireability. Feminizing job descriptions resulted in lower hireability ratings of both female and male agentic applicants. These findings held across different conditions (the manner in which applicants spoke, different business areas, subjective and objective rating scales, applicants’ parental status, and amount of information received about applicants). Our findings show that both female and male applicants are punished for overly agentic behaviour if social skills are required in a job description.

Keywords: gender stereotypes, role congruity, backlash effect, leadership, malleability of implicit stereotyping

Prof. Melanie Steffens, PhD
Department of Psychology
Friedrich Schiller University Jena
Am Steiger 3, Haus 1
07743 Jena, Germany
E-Mail: melanie.steffens@uni-jena.de

The confirmatory investigation of APM items with loadings as a function of the position and easiness of items: A two-dimensional model of APM
KARL SCHWEIZER, MICHAEL SCHREINER & ANDREAS GOLD

Abstract
The structure of APM is investigated by constraining the loadings of confirmatory factor analysis (CFA) according to item position and item easiness. The constraint of loadings according to an increasing function represents the hypothesis that the item position influences performance in completing APM items. Because of the dependency of item variance on item easiness in binary data this dependency is considered additionally. Several models with one or two latent variables associated with constant and increasing constraints that additionally reflect item easiness were applied to three subsets of APM items. A broad range of item easiness characterized two subsets whereas the range of the remaining subset was rather small. As expected, in the subsets with a broad range the model with two latent variables representing the assumed position effect and dependency did considerably better than the standard CFA model. The superiority of this model suggested that the structure underlying APM is two-dimensional.

Keywords: APM, position effect, item easiness, latent curve model, confirmatory factor analysis, fixed-links model

Karl Schweizer, PhD
Department of Psychology
Goethe University Frankfurt
Mertonstr. 17
60054 Frankfurt a. M., Germany
E-Mail: k.schweizer@psych.uni-frankfurt.de

A Reanalysis of Toomela (2003): Spurious measurement error as cause for common variance between personality factors
MATTHIAS ZIEGLER, AARO TOOMELA & MARKUS BÜHNER

Abstract
The present article reanalyzed data collected by Toomela (2003). The data contain personality self ratings and cognitive ability test results from n = 912 men with military background. In his original article Toomela showed that in the group with the highest cognitive ability, Big-Five-Neuroticism and -Conscientiousness were substantially correlated and could no longer be clearly separated using exploratory factor analysis. The present reanalysis was based on the hypothesis that a spurious measurement error caused by situational demand was responsible. This means, people distorted their answers. Furthermore it was hypothesized that this situational demand was felt due to a person’s military rank but not due to his intelligence. Using a multigroup structural equation model our hypothesis could be confirmed. Moreover, the results indicate that an uncorrelated trait model might represent personalities better when situational demand is partialized. Practical and theoretical implications are discussed.

Keywords: Personality, Faking, Situational demand, Spurious measurement error

Prof. Matthias Ziegler, PhD
Humboldt-University Berlin
Rudower Chaussee 18
12489 Berlin, Germany
E-Mail: zieglema@hu-berlin.de

Construct redundancy within the Five-Factor Model as measured by the NEO PI-R: Implications for emotional intelligence and incremental coherence
GILLES E. GIGNAC, KERRY L. JANG & TIMOTHY C. BATES

Abstract
Self-report measures of emotional intelligence (EI) have been criticized for not being associated with unique validity, independently of comprehensive measures of personality such as the NEO PI-R. In this investigation, the issue of unique validity was re-directed at personality as measured by the facets of the NEO PI-R. Specifically, based on three samples, the personality facet of Depression within the NEO PI-R was found to be so substantially predicted by ten other NEO PI-R facets as to suggest construct redundancy within the NEO PI-R (i.e., R = .93, R = .99, R = .96). Because mixed-models of EI tend to be associated with clearer construct boundaries than personality, it is suggested that EI may be associated with some scientific utility (i.e., ‘incremental coherence’), even in the total absence of any empirically demonstrable unique validity.

Keywords: personality, construct redundancy, reliability, emotional intelligence, multiple regression

Gilles E. Gignac, PhD
University of Western Australia
365 Little Collins Street, Suite 1.11
3000 Melbourne, Victoria, Australia
E-Mail: gilles.gignac@genos.com.au

IRT models with relaxed assumptions in eRm: A manual-like instruction
REINHOLD HATZINGER & THOMAS RUSCH

Abstract
Linear logistic models with relaxed assumptions (LLRA) as introduced by Fischer (1974) are a flexible tool for the measurement of change for dichotomous or polytomous responses. As opposed to the Rasch model, assumptions on dimensionality of items, their mutual dependencies and the distribution of the latent trait in the population of subjects are relaxed. Conditional maximum likelihood estimation allows for inference about treatment, covariate or trend effect parameters without taking the subjects’ latent trait values into account. In this paper we will show how LLRAs based on the LLTM, LRSM and LPCM can be used to answer various questions about the measurement of change and how they can be fitted in R using the eRm package. A number of small didactic examples is provided that can easily be used as templates for real data sets. All datafiles used in this paper are available from http://eRm.R-Forge.R-project.org/

Keywords: LLRA, Rasch-models, repeated measurements, multidimensionality, eRm

Reinhold Hatzinger, PhD
Department of Statistics and Mathematics
WU (Vienna University of Economics and Business)
Augasse 2-6
1090 Vienna, Austria
E-Mail: reinhold.hatzinger@wu-wien.ac.at