Journal article
Exploring examiner judgement of professional competence in rater based assessment
Advances in health sciences education : theory and practice, Vol.21, pp.775-788
10/2016
PMID: 26796200
Metrics
13 Record Views
UN Sustainable Development Goals (SDGs)
This output has contributed to the advancement of the following goals:
Source: InCites
Abstract
Exercise physiology courses have transitioned to competency based, forcing Universities to rethink assessment to ensure students are competent to practice. This study built on earlier research to explore rater cognition, capturing factors that contribute to assessor decision making about students’ competency. The aims were to determine the source of variation in the examination process and document the factors impacting on examiner judgment. Examiner judgement was explored from both a quantitative and qualitative perspective. Twenty-three examiners viewed three video encounters of student performance on an OSCE. Once rated, analysis of variance was performed to determine where the variance was attributed. A semi-structured interview drew out the examiners reasoning behind their ratings. Results highlighted variability of the process of observation, judgement and rating, with each examiner viewing student performance from different lenses. However, at a global level, analysis of variance indicated that the examiner had a minimal impact on the variance, with the majority of variance explained by the student performance on task. One anomaly noted was in the assessment of technical competency, whereby the examiner had a large impact on the rating, linked to assessing according to curriculum content. The thought processes behind judgement were diverse and if the qualitative results had been used in isolation, may have led to the researchers drawing conclusions that the examined performances would have yielded widely different ratings. However, as a cohort, the examiners were able to distinguish good and poor levels of competency with the majority of student competency linked to the varying ability of the student.
Details
- Title
- Exploring examiner judgement of professional competence in rater based assessment
- Creators
- Fiona L Naumann - Queensland University of TechnologyStephen Marshall - UNSW SydneyBoaz Shulruf - UNSW SydneyPhilip D Jones - UNSW Sydney
- Publication Details
- Advances in health sciences education : theory and practice, Vol.21, pp.775-788
- Publisher
- Springer Dordrecht
- Identifiers
- 991013098101502368
- Copyright
- (c) Springer Science+Business Media Dordrecht 2016.
- Academic Unit
- Faculty of Health
- Language
- English
- Resource Type
- Journal article