INNOVATIONS in pharmacy

Comparing Vertical and Horizontal Screening Methods for Pharmacy Resident Candidates Before Interviews

Michael J. Peeters

Todd E. Gundram

Julie A. Murphy

DOI: https://doi.org/10.24926/iip.v6i3.388

Keywords: interviews, screening, residency, admissions

---

Abstract

Purpose: Prior studies have examined autobiographical screening methods among medical student applicants, and demonstrated halo bias with single-rater scoring; though others have questioned its practical significance. Comparing with traditional vertical screening method, we evaluated a horizontal method for initial screening of Post-Graduate Year-1 (PGY-1) pharmacy practice resident candidate applications prior to interviews.

Methods: Our screening rubric for PGY-1 pharmacy residency candidates consisted of eight criteria, each scored using a 5-point Likert scale. During the 2014 residency recruitment season, two single-evaluators (A&B) scored all eight criteria and their scores were summed into total application scores (vertical method). Meanwhile two other evaluators (C&D) each evaluated only two criteria for all applications. The four combined-evaluators (A-D) scores, on two criteria each, were summed together into total application scores (horizontal method). For statistical comparison of single-evaluator and combined-evaluators, inter-component reliabilities were analyzed for each evaluator, while inter-rater consistency was also examined. For practical significance, actual selection differences were reviewed.

Results:Forty-six applications were evaluated to determine 24 invitations for on-site interviews. Inter-component reliability differed among evaluatorA, evaluatorB, combined-evaluators A-D (Cronbach’s alpha of 0.74, 0.73, 0.58, respectively; lower better). Among raters, inter-rater consistency was excellent (0.86 by intraclass correlation, p

Conclusion: Halo bias was seen with the single-evaluators (vertical method); two interview invitations were negatively impacted. For pharmacy resident screening, a horizontal screening method appears to be rigorous in promoting fairness for applicants. As pharmacy residency applications continue to grow, a fair and time-efficient method of screening seems imperative.

 

Type: Original Research