Validity and reliability of portfolio assessment of student competence in two dental school populations: A four-Year study

Cynthia C. Gadbury-Amyot, Michael S. McCracken, Janet L. Woldt, Robert L. Brennan

Research output: Contribution to journalArticlepeer-review

27 Scopus citations


The purpose of this study was to empirically investigate the validity and reliability of portfolio assessment in two U.S. dental schools using a unified framework for validity. In the process of validation, it is not the test that is validated but rather the claims (interpretations and uses) about test scores that are validated. Kane's argument-based validation framework provided the structure for reporting results where validity claims are followed by evidence to support the argument. This multivariate generalizability theory study found that the greatest source of variance was attributable to faculty raters, suggesting that portfolio assessment would benefit from two raters' evaluating each portfolio independently. The results are generally supportive of holistic scoring, but analytical scoring deserves further research. Correlational analyses between student portfolios and traditional measures of student competence and readiness for licensure resulted in significant correlations between portfolios and National Board Dental Examination Part I (r=0.323, p<0.01) and Part II scores (r=0.268, p<0.05) and small and non-significant correlations with grade point average and scores on the Western Regional Examining Board (WREB) exam. It is incumbent upon the users of portfolio assessment to determine if the claims and evidence arguments set forth in this study support the proposed claims for and decisions about portfolio assessment in their respective institutions.

Original languageEnglish (US)
Pages (from-to)657-667
Number of pages11
JournalJournal of dental education
Issue number5
StatePublished - May 1 2014


  • Assessment
  • Competency-based education
  • Critical thinking
  • Dental education
  • Educational measurements
  • Multivariate generalizability theory
  • Portfolio assessment
  • Problem-solving
  • Self-assessment

ASJC Scopus subject areas

  • Education
  • Dentistry(all)


Dive into the research topics of 'Validity and reliability of portfolio assessment of student competence in two dental school populations: A four-Year study'. Together they form a unique fingerprint.

Cite this