This resource provides information on a variety of assessment methods that are frequently used in health professional education1.
Purposes of assessment
The main purpose of assessment is to obtain formative or summative information on students’ progress. Other purposes are to obtain diagnostic information for placement purposes, to monitor throughput rates, to inform external stakeholders and for certification purposes.
Formative assessments are used to provide students with feedback on their progress and to inform lecturers whether students have mastered course material. Formative assessments are typically not awarded marks. Rather, global scores of mastery, oral or written feedback is provided. Formative assessments are typically situated within courses, not at the end of the course. Formative assessment should not be conflated with continuous assessments, where the marks that are allocated count towards the students’ year marks.
Summative assessments are used to determine students’ progress by the end of a course. Pass/fail decisions are based on marks obtained in summative assessments.
Principles of assessment
Norm- and criterion-referenced assessment methods2,5
In assessment theory, a distinction is made between norm-referenced and criterion-referenced assessments. The difference between the two theoretical perspectives is important as they yield different types of information about students’ progress, they have different purposes and they are marked or scored differently. In health professional education, criterion-referenced assessments are frequently used in clinical or work-based settings where the assessment focus is on students’ performance. Norm-referenced assessments are typically used in classroom settings where factual or theoretical knowledge is assessed.
Norm-referenced assessment methods are more traditional and are based on psychometric theory. In criterion-referenced assessments, student performance is judged on assessment criteria linked to outcomes or competencies. As pre-specified assessment criteria are explicit and given to the student up front, criterion-referenced methods are more transparent.
Norm-referenced and criterion-referenced assessments are not mutually exclusive. Both approaches are used in assessment practice and both contribute to the decisions regarding students’ levels of competence.
In medical education concerns of validity and reliability of the assessment method are always important. A valid assessment should measure what it intend to measure or was designed to measure. Validity refers to the degree that the evidence supports that the test interpretations are correct.
Reliability refers to the consistency of assessment and the ability of results to be generalized to other performances. Reliability implies that the same results will be obtained when the assessment is administered to a student under the same or similar conditions.
Standard setting is concerned with setting a cut off score that separates the competent from the non-competent student on a test or performance based assessment. It is a process that involves agreement by experts in the course or subject what constitutes competent versus non-competent taking into account the content taught and year of study 1.
There are various approaches in standard setting for both written and performance based assessment to support a fair process in coming to an agreement of what constitutes good enough to be considered competent 3.
The references below discusses the different approaches used in standard setting which can help the lecturer to make an informed choice of which method to use that best supports the selected assessment method 2.
Frequently used assessment methods
Norm-referenced assessment methods
1. Constructing Written Test Questions for the Basic and Clinical Sciences, 3rd revision. (2002). Contributing authors: Case, S.M. and Swanson, D. B. National Board of Medical Examiners (NBME). Printed in the United States of America.
2. Gaytan, J & McEwen, B. (2003). Effective online instructional and assessment strategies. American Journal of Distance Education, 21(3), 117–132
3. Shumway, J.M. & Harden, R.M. (2003) AMEE Guide no. 25: The assessment of learning outcomes for the competent and reflective practitioner. Medical Teacher, 25, 6, 569-584
4. Norcini, J. & Burch, V. (2007). Workplace-based assessment as an educational tool: AMEE Guide No. 31. Medical Teacher 29, 855-871
5. Newble, D. (2004) Techniques for measuring clinical competence: Objective Standard Clinical Examinations. Medical Education, 38, 199-203 (Blueprint)
6. Burch, V. & Seggie, J. (2005). Portfolio assessment using a structured interview. Medical Education, 39, 114.
7. Victoria University Teaching Development Centre. (2004). Improving Teaching and Learning: Group Work and Group Assessment. Victoria University of Wellington, Australia.
8. Finn, G.B. & Garner, J. (2011). Twelve tips for implementing a successful peer assessment. Medical Teacher, 33, 443–446
9. Colthart, I., Bagnall, G., Evans, A., Allbutt, H., Haig, A., Illing, J. & McKinstry, B. (2008). The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10. Medical Teacher, 30, 124–145