1. Informed Assessment Approaches
These resources will describe common concepts and debunk misconceptions about testing, assessment, and evaluation in speech-language pathology and audiology graduate programs. They will help you to understand your own biases, explore self-assessment and its limitations, and support informed assessment approaches.
Prior to learning about the process of assessment, it is important that each supervisor / preceptor understands their own biases about supervisees from different cultures.
The first step to this is to complete the Harvard Implicit Bias Test. Complete tests related to race, gender, transgender, disability, religion, or a variety of other implicit biases that we may have.
Documents & Articles (Read)
Assessment in Medical Education (New England Journal of Medicine)
This article provides a conceptual framework for and a brief update on commonly used and emerging methods of assessment discusses the strengths and limitations of each method and identifies several challenges in the assessment of physicians' professional competence and performance.
Assessment in Medical Education
Differences in Narrative Language in Evaluations of Medical Students by Gender and Under-represented Minority Status (NIH)
This article discusses ways in which narrative evaluations are different for women and underrepresented minorities. These groups were more likely to be evaluated based on personal attributes rather than their competency. The article provides ways to reduce these biases in evaluations.
Differences in Narrative Language in Evaluations of Medical Students
Assessment may not only drive learning, it may also help learning (Timothy Wood)
The author reviews two articles regarding the effect of assessment on learning retention. He also explores some negative effects of testing and raises questions gleaned from the literature review.
Assessment may not only drive learning, it may also help learning
Test-enhanced learning in medical education (Douglas P Larsen, Andrew C Butler, Henry L Roediger III)
Tests not only assess the efficacy of curricula and assign grades but also enhance learning by promoting better retention of information, known as the testing effect. In medical education, frequent and spaced-out tests with effortful recall and feedback can strengthen clinical knowledge and expertise.
Test-enhanced learning in medical education
Tensions in Informed Self-Assessment: How the Desire for Feedback and Reticence to Collect and Use It Can Conflict (Dr. Karen Mann et al)
This paper explores the tensions described by learners and professionals when informing their self-assessments of clinical performance.
Assessment in the post-psychometric era: learning to love the subjective and collective (Brian Hodges)
As you are reviewing this information, consider how psychometric tests may not be an inclusive method for certain students you supervise.
Assessment in the post-psychometric era: learning to love the subjective and collective
Videos (Watch)
Measurement, Assessment, and Evaluation in Clinical Education (CAPCSD)
This video will help define these important terms and help you to differentiate them and their roles in clinical education. (transcript)
2: Purposes of Assessment
These resources will explore and describe the purposes of assessment. You will review several resources that will explain the role of assessment in supporting competency in clinical education, motivating learners, providing feedback, communicating what is important, and facilitating learning. You will also explore the concept of equity in assessment and the impact that inequitable assessment can have on clinical learners.
Documents & Articles (Read)
Debriefing as Formative Assessment: Closing Performance Gaps in Medical Education (Jenny W. Rudolph PhD, Robert Simon EdD, Daniel B. Raemer PhD, Walter J. Eppich MD, MEd)
The authors present a four-step model of debriefing as formative assessment that blends evidence and theory from education research, the social and cognitive sciences, experience drawn from conducting over 3,000 debriefings and teaching debriefing to approximately 1,000 clinicians worldwide. Also, the authors propose that the model, designed for post-simulation debriefings, can also be applied to bedside teaching in the emergency department and other clinical settings.
Debriefing as Formative Assessment: Closing Performance Gaps in Medical Education
MEdIC Series: The Case of the Failure to Fail (Tamara McColl)
In season 4, episode 8 of the ALiEM Medical Education in Cases (MEdIC) series, the case of an attending emergency physician who struggles with the ingrained “failure to fail” culture amongst his colleagues when faced with a resident who has significantly underperformed throughout his emergency medicine rotation is presented. Also, find a hosted discussion around this case with insights from the ALiEM community in Expert Review and Curated Community Commentary.
MEdIC Series: The Case of the Failure to Fail
Defining and Assessing Professional Competence (Ronald M. Epstein, MD, and Edward M. Hundert, MD)
In this article, the authors advance a definition of professional competence in medical education, perform an evidence-based critique of current methods of assessing 6 key areas of competence, and propose new means for assessing residents and medical students. Core concepts in this discussion are pertinent to audiology and speech-language pathology as those fields move to competency-based education and assessment.
Defining and Assessing Professional Competence
Medical Education’s Wicked Problem: Achieving Equity in Assessment for Medical Learners (Lucey, Catherine R. MD; Hauer, Karen E. MD, Ph.D.; Boatright, Dowin MD; Fernandez, Alicia MD)
In this article, the authors review the underlying individual and structural causes of inequity in assessment. Using an organizational model, they propose strategies to achieve equity in assessment.
Medical Education’s Wicked Problem: Achieving Equity in Assessment for Medical Learners
Assessment may not only drive learning, it may also help learning (Timothy Wood)
The author reviews two articles regarding the effect of assessment on learning retention. He also explores some negative effects of testing and raises questions gleaned from the literature review.
Assessment may not only drive learning, it may also help learning
Videos (Watch)
The role of clinical assessment in competency-based medical education (Holmboe, E. S., et al)
How clinical assessment in medical education improves clinical skill development and overall competency for healthcare trainees.
3. Common Assessment Methods in Health Professions Education
These resources will help you to distinguish between common assessment methods in health professions education. They describe the skills, knowledge, and professionalism framework for assessment in addition to describing workplace-based assessment. They will help you to identify tools used for assessing skills, knowledge, professionalism, and workplace-based learning.
Documents & Articles (Read)
Conveying Expectations about Professional Behavior, Audiology Today
The author explores fundamental expectations of professional behavior essential to the ethical practice of audiology and speech-language pathology.
Conveying Expectations about Professional Behavior
Assessment Methods in Medical Education, International Journal of Health Sciences
Medical schools have changed as teachers have progressed from the role of problem-identifier to that of the solution provider. To address these changes, medical schools have developed new curricula, introduced new learning situations, introduced new methods of assessment, and realized the importance of staff development. This resource outlines the variety of assessment methods utilized in medical education.
Assessment Methods in Medical Education
Portfolios for assessment and learning: AMEE Guide no. 45, Jan Van Tartwijk and Erik W. Driessen
The authors explore portfolios as tools for assessment of performance in the workplace as well as tools to stimulate learning from experience. They provide an overview of the content and structure of various types of portfolios, describe the potential of electronic portfolios, present techniques, and strategies for using portfolios as tools for stimulating learning and for assessment, and discuss factors that influence the success of the introduction.
Portfolios for assessment and learning: AMEE Guide no. 45
Assessment of clinical competence using objective structured examination, R M Harden, M Stevenson, W W Downie, and G M Wilson
In this resource, the authors introduce the structured clinical examination in which the variables and complexities of the examination are more easily controlled, its aims can be more clearly defined, and more of the student’s knowledge can be tested. The examination results in improved feedback to students and staff. Although this is an older resource, the information contained is still relevant to current clinical practice.
Assessment of clinical competence using objective structured examination
Videos (Watch)
The Seven Step Method of PBL, Aalborg University
Watch this video to explore how to use the Problem Based Learning model
Assessment Drives Learning, Cees van der Vleuten
Watch this video to explore how assessment can be conducted in problem-based learning.
What Research Has to Say about Assessment, Cees van der Vleuten
Watch the first six minutes and thirty seconds of this symposium on medical education to learn about what research has to say about assessment.
4: Establishing Assessment Quality
These resources will explore working definitions of reliability, validity, and authenticity. You will identify practical considerations that are important to establishing assessment quality and identify the role and training needs of assessors.
Documents & Articles (Read)
Measurement Principles in Medical Education, Chapter 1 (pp. 8-15) (Ann Burke)
This chapter discusses the basic concepts of validity, reliability, and utility of assessment methods in medical education. It discusses the various considerations for and threats to validity, reliability, and authenticity of assessment tools and processes; and introduces the readers to Miller’s Pyramid, a framework for assessing clinical competence at four different levels (knows, knows how, shows, and does).
Measurement Principles in Medical Education
Seeing the 'black box' differently: assessor cognition from three research perspectives, (Linda Snell)
Review this KeyLIME abstract that explores three distinct perspectives on assessor cognition - the assessor as trainable, the assessor as fallible, and the assessor as meaningfully idiosyncratic. You can also access the podcast here in which Linda Snell discusses the paper by Gingerich et al. in more depth. Please note that these authors use the term “assessors” rather than “raters” as a more general term to include qualitative and quantitative measures.
Seeing the 'black box' differently: assessor cognition from three research perspectives
Assessment and Bias (DePaul University)
There are many ways biases, both explicit and implicit, can threaten the validity, reliability, and authenticity of the assessment tools and the assessment process. This webpage from DePaul University’s Teaching Commons discusses confirmation bias and stereotype threat to assessment validity.
Uncovering Implicit Bias in Assessment, Feedback, and Grading (Anthony Reibel)
This article provides examples of strategies to mitigate assessment/feedback bias in education.
Uncovering Implicit Bias in Assessment, Feedback, and Grading
Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review (Kogan, J., Holmboe, E., Hauer, K.)
Direct observation of medical trainees with actual patients is important for performance-based clinical skills assessment. Multiple tools for direct observation are available, but their characteristics and outcomes are reviewed.
Assessment in Medical Education (Epstein, R.)
This article in the Medical Education series provides a conceptual framework for and a brief update on commonly used and emerging methods of assessment, discusses the strengths and limitations of each method, and identifies several major challenges in assessing professional competence and performance.
Videos (Watch)
Introduction to Reliability and Validity, Scott Branson
This video provides a basic overview of validity and reliability as they relate to instrument development.
Interview on Experimental Evidence on Teachers' Racial Bias in Student Evaluation: The Role of Grading Scales, David Quinn.
In this interview by a local news channel, David Quinn explains his study (2020) that showed teacher’s racial bias while grading students’ written work, as the potential strategy to reduce the impact of such bias.
Interview on Experimental Evidence on Teachers' Racial Bias in Student Evaluation: The Role of Grading Scales
5: Assessment as a Driver of Learning
Through these resources, you will be encouraged to reflect on how you deliver clinical education and the ways in which you assess the learner’s competency as a result of that education. The concepts of competency-based education and competency-based assessment will be explored.
Documents & Articles (Read)
Assessment in Medical Education; What Are We Trying to Achieve? (Helena Ferris & Dermot O’Flynn)
This article highlights challenges of assessment, such as which type of assessment to use and when, and outlines current trends in assessment. Self-assessment and peer assessment have become popular in education and can be effective additions to traditional assessment methods. As you review this article, consider the methods of assessment you use with student clinicians and how these assessments may (or may not!) reinforce students’ motivation to learn. Are your assessments effective?
Assessment in Medical Education; What Are We Trying to Achieve?
Foundations of quality in competency-based programs: Competencies and assessments, Conna Bral, Jennifer Cunningham
This article provides an overview of the history and expectations of competency-based assessment.
Foundations of quality in competency-based programs: Competencies and assessments
A powerful way to improve learning and memory, Jeffrey D. Karpicke
In this article, the author explores the impact of memory retrieval on learning and investigates the benefits of retrieval as a learning event. He summarizes his findings with four “Take Home Points About Retrieval-based Learning.” These take-home points may give you a starting point in assisting a student in applying theoretical knowledge from the classroom to clinical practice.
A powerful way to improve learning and memory
The Role of Formative Assessment in Effective Learning Environments, D. Wiliam
Formative assessment is vital for our students, but how should we provide it? And when? How does formative assessment drive student learning? This chapter excerpt highlights that students learn best when they are engaged in learning and that the environment in which they learn is important. Focus particularly on the section Formative Assessment as Feedback on p. 137 and Theoretical Syntheses, starting on p. 147. An important concept starts on p. 152 and is titled Formative Assessment and the Regulation of Learning Processes.
The Role of Formative Assessment in Effective Learning Environments
Assessment Not Only Drives Learning, It May Also Help Learning, Timothy Wood
This article provides a brief introduction to testing effects and the transfer of learning from written tests to clinical application.
Assessment Not Only Drives Learning, It May Also Help Learning
Countering Bias in Assessment
For another perspective on the assessment of learners, this is a very short article with straightforward suggestions on minimizing bias in your supervisory interactions.
Twelve Tips for programmatic assessment (C.P.M. van der Vleuten , L.W.T. Schuwirth , E.W. Driessen , M.J.B. Govaerts & S. Heeneman)
In this resource, the authors provide concrete recommendations for implementation of programmatic assessment, which refers to the design of an assessment program with the intent to optimize its learning function, its decision-making function, and its curriculum quality-assurance function.
Twelve Tips for programmatic assessment, C.P.M. van der Vleuten
Videos (Watch)
The Role of Clinical Assessment (TomorrowsMed)
This video is a conversation between medical educators about the learning that comes from and to the student, clinical educator, and the patient when formative assessment occurs in training. The panel also discuss the role of competency-based assessment in teaching and learning.
SSATNC14: Why can students pass tests but fail at life? (Eric Mazur)
Eric Mazur, Balkanski Professor of Physics and Applied Physics at Harvard University, discusses how students can pass tests but still prove unsuccessful in life. Feel free to watch the entire video but you should watch the presentation until at least 17:42 to think about the development of assessment questions.
SSATNC14: Why can students pass tests but fail at life?
A Journey Towards Programmatic Assessment, (Department of Medical Education)
Watch Cees van der Vleuten as he gives a lecture on programmatic assessment in May 2015 at the annual Cudmore Lecture. Start watching this clip at 36:56.
A Journey Towards Programmatic Assessment
Competency-Based Education, CUW Online
This video addresses three questions: What is competency-based assessment, What are the affordances of competency-based assessment, and How can I leverage competency-based assessment in my own learning environments? It highlights the benefits of a competency-based approach and how this type of assessment can impact student learning.