Self-serving bias on job analysis ratings
US research, June 2013
A number of studies have identified self-serving bias as a process by which individuals (for example recruiters or supervisors) over-rate the importance of their personality characteristics as critical to successful performance in a particular job. Correspondingly, characteristics which a recruiter or supervisor does not possess are under-weighted or ignored as critical for success. Could it also be that this same self-serving bias influences not only the way a job description is interpreted by a recruiter or supervisor, but even the development of the job description itself?
This is the question members of the US Office of Personnel Management, the National Security Agency and Customs and Boarder Protection asked via the examination of a large scale multi-agency job analysis survey. The affirmative answer speaks to the need for Job Analysis experts to be made aware of the tendency for self-serving bias to influence their decisions, as well as the importance of diverse teams having input into a critical moment in the employment life cycle, namely job creation. The implications for self-serving bias clearly extend to the evaluation of critical job competencies at all stages along the employment life cycle, and in particular promotion decisions.
This research aimed to identify whether self-serving rater bias influences the job analysis process, and in particular whether an incumbent’s competency on certain job characteristics will positively influence the importance of that characteristic according to the incumbent.
Drs Cucina (US Customs and Border Protection), Martin (US Office of Personnel Management), Vasilopoulos (National Security Agency) and Thibodeuax (US Office of Personnel Management) reviewed archival multi-agency government wide survey data of 57 clerical and technical occupations, with 26,682 incumbents. The incumbents had been asked to rate themselves against a range of 31 competencies (e.g. reading, writing, speaking, listening, mathematical reasoning and decision-making) on a 5 point scale (from Unsatisfactory to Outstanding) as well as the importance of that competency for their job (from Not important to Extremely important). The original survey results were to be used to determine critical competencies for successful performance in technical and clerical occupations, and therefore selection criterion.
The job analysis ratings were standardised to take into account any job specific differences, for example, if maths was a critical competency for an occupation then it would attract those with maths capabilities which would be reinforced by self-selection bias. Standardisations by the researchers were noted in the data analysis.
The primary finding was that in relation to all of the measured competencies, incumbents’ self-ratings of competency were positively (and significantly) correlated with importance ratings. The correlations ranged from .239 to .432, with a median value of .342 indicating that self-rater competency bias had a “moderate” effect on importance ratings. This means that incumbents were more likely to rate the importance of a competency as high if that was a competency that was a personal strength, and correspondingly to rate low the importance of a competency which was a personal weakness. Moreover this self-rater bias accounted for a 6-19% variation in ratings between raters, i.e. inconsistencies did not reflect accurately or objectively the different value of competencies but the subjective value attributed by raters.
This study has important implications for the ways in which job competencies are designed or described, as well as the job/promotion selection process. It seems clear that recruiters and supervisors are prone to self-serving bias and this is reflected in the emphasis given to certain job characteristics (i.e. those which are a recruiter’s/supervisor’s strength) and the minimisation of others. In this way, recruitment and promotion decisions are likely to be influenced by a decision-maker’s competencies rather than solely by the needs of the job. Recognising this bias is the first step to remediation, but awareness is unlikely to be sufficient. Ensuring that recruitment and promotion decisions (as well as competency frameworks) are made by a diverse group is key, and this study points to the importance of perspective diversity based on competency diversity. In practical terms this may mean that recruitment and promotion selection panels should be comprised of individuals with diverse educational backgrounds, or from diverse work functions, it certainly means that reliance upon one decision-making individual is likely to result in a unconsciously biased outcome.
To read the full article, see Cucina, J. M., Martin, N. R., Vasiloploulos, N. L., and Thibodeuax, H. F., (2012) “Self-serving bias effects on job analysis ratings” The Journal of Psychology: Interdisciplinary and Applied, Vol 146, No 5, 511-531.