Table 2 Characteristics of programmes...

Table 2

Characteristics of programmes

Study	Population, teaching format and context	Instructional methods	Programme content	Programme outcomes	Key findings and educational impact
1. Cross et al. (2001)	Population: Physiotherapists (n = 44)	Didactic teaching of assessment concepts Simulated assessment practice, scoring six videos of different levels of performance	Key features of effective assessment, including formative and summative assessment	Correlation between CEs and academic ratings pre- and post- training	Key findings: Pre-training correlations showed a wide difference in CEs' assessment scores. CEs' grades aligned more with academics for the lower-level students, with variability at mediocre and good levels 57% of CEs showed stronger standardized assessment scores after training irrespective of the assessment instrument. General tendency to regress to the mean, resulting in poorer students' performance not receiving a fail grade Most educators (86%) preferred Form B Educational impact: Training impact was variable, demonstrating mostly positive, a small number of negative and in some cases no influence on CEs' assessment decisions
	Format: In-person training programme, 2.5 h duration		Explicit criteria for good/bad performances	Comparison of CE mean ranking of student's level of performance with standardised rankings of the group
	Context: Single training occasion, University setting		Simulated assessment practice pre-post training	Utility of two assessment tools pre- and post-training and rationale for preference between tools
2. Vendrely and Carter. (2004)	Population: Physical Therapists (n = 34) Format of CPI: In-person training, 1 h duration Context CPI: Single training occasion, University setting Format of CIECP: In-person training, 15 h duration Context CIECP: Training occasions not provided, University setting	CPI – Didactic teaching of assessment processes	CPI – familiarisation with assessment criteria	Correlations of CEs’ ratings of a 12-min video using five key assessment criteria from the CPI assessment tool (criteria included safety, responsible behaviour, ethical practice, professional behaviour, legal practice)	Key findings: For safety, CIECP & CPI training was rated lower than CIECP only, CPI only, or no training. Post hoc analysis for responsible behaviour showed a significant difference between CIECP & CPI and CPI only training (p = 0.005), and CIECP & CPI training and no training (p = 0.045). Only half of the participants rated ethical practice or legal practice, which raised concerns but may have been related to limited information provided in the video performance Educational impact: Additional research required on assessment instruments, in particular validity and reliability
		Simulated assessment practice using assessment criteria and written description of a student's behaviour	Simulated grading practice. Additional reading of glossary items and sample scoring
		CIECP – no details of teaching methods used	CIECP – roles and responsibilities in clinical education, basic learning theory, teaching methods, performance assessment, managing an exceptional student, legal issues
3. Ferguson et al. (2010)	Population: Speech and Language Pathologists (n = 240) Format: In-person training, 1.5–2 h duration Context: Single training occasion, University setting	Didactic teaching of assessment concepts/processes	Module 1: Introduction to COMPASS concepts and processes	Qualitative analysis of learning goals pre- and post-training for evidence of educational concepts and new assessment approaches % of generic and occupational units of competencies evidenced in the learning goals pre- and post-training Evidence of PBA concepts - direct observation & VAS to show incremental development of competence	Key findings: Small positive shift in assessment language used in learning goals post-workshop. Overall, there was a similar proportion of main assessment concepts pre- and post-workshop, the exception being formative assessment, which increased from 6% to 15%
		Learning goals created for a written student scenario	Module 2: Assessment as learning		Little change in numbers of generic or occupational units of competency (evidence of assessment concepts) in learning goals
		Provision of two short written student cases to gauge CEs' understanding of new assessment tool	Module 3: Implementing the assessment tool to assess and support marginal students		Rating scale for entry-level descriptions and the need for direct observation in PBA were either partially or fully evident from most participants. 10% of participants did not reflect the need for direct observation in PBA, which may signpost the need for more training
		Workshops with facilitated large and small group activities	Simulated assessment practice		Educational impact: Training workshops assist CEs’ familiarisation with the new assessment tool and the use of a formative assessment approach
4. Durham et al. (2012)	Population: Nurses (n = 6) Format: In-person training, 6 h duration Context: 2 training occasions, location not reported	Didactic teaching of assessment concepts/SOM role	Exploration of SOM's role versus traditional mentor, NMC standards, accountability and assessment	Evaluation of all aspects of the pilot programme using a six-point Likert scale (1 not useful – 6 very useful) Feedback identifying the least/most beneficial aspects of the programme SOM role assessments - OSCE exams with three scenarios of students and five OSCE stations	Key findings: All aspects of programme positively rated on Likert scale, but medians/IQR not reported Most beneficial aspect of programme: meeting the educational team, understanding the role of the SOM, familiarisation with assessment documents, structure of sessions, ability to pass two SOM assessments, presentations, role play/OSCE, discussion. Least beneficial aspects: timetabling, insufficient programme detail, scheduling of OSCE stations, role play. Frequencies were not provided Authors report OSCEs and student scenarios exams showed participants had a good knowledge of the SOM role Educational impact: The training programme was effective in preparing SOM to undertake the final clinical assessments with student nurses
		Group discussion and reflection activities	Supporting failing students and those with additional needs
		Review of practice case scenarios	Provision of effective feedback
		Generation of portfolios with evidence of learning and mock action plans	Recourses to support SOMs
		Peer support group	Review of practice case scenarios
5. Bacon et al. (2015)	Population: Dietitians (n = 9) Format: Web-based self-directed learning, 7 h duration Context: Multiple contacts, online programme	Teaching and learning material delivered online	Learning outcomes and development of competency-based assessment skills Competency standard, assessment context and applying entry-level competency standards in clinical settings Simulated assessment practice to consolidate credible and defensible practice Comparison of grades with experienced supervisor	Rating of CEs' confidence levels assessing student competence pre- and post-training (1–10 numerical rating scale) Rating one video of a student's performance pre- and post-training Feedback from focus group/phone interviews, including how CEs ensure credible and defensible assessments Multiple-choice knowledge quiz post-training	Key findings: Pre-programme mean confidence rating 5.57 (range 2–9). No post-programme ratings available due to technical issues
		Provision of reading and video learning resources			Five participants (63%) rated student's performance as intermediate/beginner consistent with experienced supervisors. No post-programme ratings available due to technical issues
		Problem-based learning activities			Written feedback (n = 4/9, 44%) positive for (i) developing CEs' confidence to supervise students, (ii) utility of videos to understand performance and (iii) accessibility, utility and interdependence of teaching strategies
		Simulated assessment practice with 20 videos of students' performances available			Average score for quiz 86%
		Online peer discussion forum			Educational impact: Online education programme is a viable solution to develop clinical assessment skills and confidence. Allows access across a wide geographical community
6. Bacon et al. (2017)	Population: Dietitians (n = 26)	Didactic teaching of assessment concepts/processes	Review of recently revised assessment competencies Simulated assessment practice with opportunity to change rating and confidence scores after group discussion	Rating one video of a student's performance using VAS (1 novice −7 competent) and comparison of within-group ratings pre- and post-training	Key findings: No consistency in rating students' performances pre- or post-training. Pre-training median 4 (range 1–6) (n = 11/26), post-training median 4 (range 2–6) (n = 16/26) 35% (n = 9/26) of participants changed ratings to 4 after the training. No expert ratings were provided for video, therefore difficult to interpret the impact of these changes 31% (n = 8/26) increased assessment confidence, 27% (n = 7/26) reduced assessment confidence and 42% were unchanged (n = 11) post-training Qualitative themes identified (i) assessment discussion supports CEs' decisions, (ii) peer discussion during assessment leads to more holistic judgements and (iii) multiple sources of evidence/reflections are required for trustworthy assessment decisions Educational impact: Workplace assessment is a matter of consensus. Discussing judgements with colleagues can assist with a shared mental model of competence. The authors suggest that a programmatic assessment approach warrants further review for high-stakes workplace exams
	Format: In-person training, duration not specified	Workshop with simulated assessment practice using one video of a student's performance		Rating of CEs' confidence levels assessing a student's performance (numerical 1–10 rating scale)
	Context: Single training occasion, location not reported	Facilitated group discussion including sharing individual assessment grades and reflection on grades awarded		Global description of a student's overall performance
7. Moline et al. (2018)	Population: Radiation Therapists (n = 24)	Workshop with simulated assessment practice using one video of student's performances Facilitated group work	Simulated assessment practice using the standard evaluation tool Facilitated debriefing of grades awarded for simulated assessment practice and review of assessment criteria	Rating CEs' confidence levels assessing a student's performance using a Likert scale 1–5 (1 low - 5 high) pre- and post-training Rating CEs' confidence at 75 days post-training using the same scale (Likert 1–5)	Key findings: CEs' confidence rating a student's performance was significantly higher post-Workshop B compared to pre-workshop and at 75 days post- training (p < 0.05)
	Format: In-person training, 1 h duration				Educational impact
	Context: Single training occasion, tertiary clinical care centres				A short 1-h training workshop improves CEs' confidence to undertake competency assessments with the training effect persisting for at least 75 days
8. Alpine et al. (2021)	Population: Physiotherapists (n = 53)	Didactic teaching of assessment concepts	Review of assessment criteria and rating scale Introduction to FoRT principles Simulated assessment practice of ‘poor’ and ‘very good’ performances Facilitated group reflection and discussion of grades awarded for simulated assessment practice with an opportunity to change grades after group discussion Common pitfalls of assessment incorporated into group discussion	Rating videos of two different levels of students' performances pre- and post-group discussion against expert score Impact of demographics, professional experience, student supervision experience and professional grade on grading students	Key findings: Video 1 (poor performance): Alignment with expert ratings increased from 26% pre- discussion to 48% post discussion; 22% decreased scores from 4 (pass) to 3 (fail). For Video 2 (very good performance): Alignment with expert scores decreased from 43% pre-discussion to 41% post-discussion; 25% of participants lowered scores post discussion
	Format: In-person training, 1.5 h duration	Workshops based on Frame of Reference Training (FoRT) principles using simulated assessment practice with two videos of two different levels of performance			No significant difference based on gender and years qualified. Professional grade showed a difference for one learning outcome only. In terms of correlations, there was no significant relationship between supervision experience or the number of students supervised and change in scores
	Context: Single training occasion, University setting	Facilitated group discussion including sharing individual grades, CEs' rationale for same and reflection on grading			Educational impact: Significant changes in CEs' grades after group discussion, particularly for borderline pass/fail students, and improved alignment with expert ratings. Positive benefits of FoRT to assist decision-making in PBA.
9. Milne et al. (2022)	Population: Physiotherapists (n = 109)	Didactic teaching of assessment concepts/processes	Reflection/learning activities to guide understanding of entry-level practice thresholds Consideration of personal biases Simulated assessment practice of a student's performances Grade calibration through peer discussion of grades awarded during simulation assessment practice moderated by facilitators Calibration of assessment standards by identifying appropriate behaviours for each APP domain	Rating CEs' knowledge, skills, attributes (22 items) and confidence (six statements) using a five-point Likert scale (1 disagree – 5 strongly agree) Appraisal of CEs' perception of their ability to assess a student's performance using the APP.	Key findings: Post-training, statistically significant change in mean scores related to perceived knowledge, skills and attributes, relevant to student assessment, and overall confidence undertaking an assessment using APP (p < 0.001)
	Format: In-person training, 5 h duration	Workshop using simulated assessment practice with one video of a student's performance, followed by group discussion including rationale for ratings			Educational impact: A one-day in-person workshop significantly improved knowledge, skills, attributes and confidence to assess a student's performances
	Context: Single training occasion, University setting	Facilitated group discussion and reflection			The workshop assisted CEs' understanding of the assessment tool and competency standards

Study	Population, teaching format and context	Instructional methods	Programme content	Programme outcomes	Key findings and educational impact
1. Cross et al. (2001)	Population: Physiotherapists (n = 44)	Didactic teaching of assessment concepts Simulated assessment practice, scoring six videos of different levels of performance	Key features of effective assessment, including formative and summative assessment	Correlation between CEs and academic ratings pre- and post- training	Key findings: Pre-training correlations showed a wide difference in CEs' assessment scores. CEs' grades aligned more with academics for the lower-level students, with variability at mediocre and good levels 57% of CEs showed stronger standardized assessment scores after training irrespective of the assessment instrument. General tendency to regress to the mean, resulting in poorer students' performance not receiving a fail grade Most educators (86%) preferred Form B Educational impact: Training impact was variable, demonstrating mostly positive, a small number of negative and in some cases no influence on CEs' assessment decisions
	Format: In-person training programme, 2.5 h duration		Explicit criteria for good/bad performances	Comparison of CE mean ranking of student's level of performance with standardised rankings of the group
	Context: Single training occasion, University setting		Simulated assessment practice pre-post training	Utility of two assessment tools pre- and post-training and rationale for preference between tools
2. Vendrely and Carter. (2004)	Population: Physical Therapists (n = 34) Format of CPI: In-person training, 1 h duration Context CPI: Single training occasion, University setting Format of CIECP: In-person training, 15 h duration Context CIECP: Training occasions not provided, University setting	CPI – Didactic teaching of assessment processes	CPI – familiarisation with assessment criteria	Correlations of CEs’ ratings of a 12-min video using five key assessment criteria from the CPI assessment tool (criteria included safety, responsible behaviour, ethical practice, professional behaviour, legal practice)	Key findings: For safety, CIECP & CPI training was rated lower than CIECP only, CPI only, or no training. Post hoc analysis for responsible behaviour showed a significant difference between CIECP & CPI and CPI only training (p = 0.005), and CIECP & CPI training and no training (p = 0.045). Only half of the participants rated ethical practice or legal practice, which raised concerns but may have been related to limited information provided in the video performance Educational impact: Additional research required on assessment instruments, in particular validity and reliability
		Simulated assessment practice using assessment criteria and written description of a student's behaviour	Simulated grading practice. Additional reading of glossary items and sample scoring
		CIECP – no details of teaching methods used	CIECP – roles and responsibilities in clinical education, basic learning theory, teaching methods, performance assessment, managing an exceptional student, legal issues
3. Ferguson et al. (2010)	Population: Speech and Language Pathologists (n = 240) Format: In-person training, 1.5–2 h duration Context: Single training occasion, University setting	Didactic teaching of assessment concepts/processes	Module 1: Introduction to COMPASS concepts and processes	Qualitative analysis of learning goals pre- and post-training for evidence of educational concepts and new assessment approaches % of generic and occupational units of competencies evidenced in the learning goals pre- and post-training Evidence of PBA concepts - direct observation & VAS to show incremental development of competence	Key findings: Small positive shift in assessment language used in learning goals post-workshop. Overall, there was a similar proportion of main assessment concepts pre- and post-workshop, the exception being formative assessment, which increased from 6% to 15%
		Learning goals created for a written student scenario	Module 2: Assessment as learning		Little change in numbers of generic or occupational units of competency (evidence of assessment concepts) in learning goals
		Provision of two short written student cases to gauge CEs' understanding of new assessment tool	Module 3: Implementing the assessment tool to assess and support marginal students		Rating scale for entry-level descriptions and the need for direct observation in PBA were either partially or fully evident from most participants. 10% of participants did not reflect the need for direct observation in PBA, which may signpost the need for more training
		Workshops with facilitated large and small group activities	Simulated assessment practice		Educational impact: Training workshops assist CEs’ familiarisation with the new assessment tool and the use of a formative assessment approach
4. Durham et al. (2012)	Population: Nurses (n = 6) Format: In-person training, 6 h duration Context: 2 training occasions, location not reported	Didactic teaching of assessment concepts/SOM role	Exploration of SOM's role versus traditional mentor, NMC standards, accountability and assessment	Evaluation of all aspects of the pilot programme using a six-point Likert scale (1 not useful – 6 very useful) Feedback identifying the least/most beneficial aspects of the programme SOM role assessments - OSCE exams with three scenarios of students and five OSCE stations	Key findings: All aspects of programme positively rated on Likert scale, but medians/IQR not reported Most beneficial aspect of programme: meeting the educational team, understanding the role of the SOM, familiarisation with assessment documents, structure of sessions, ability to pass two SOM assessments, presentations, role play/OSCE, discussion. Least beneficial aspects: timetabling, insufficient programme detail, scheduling of OSCE stations, role play. Frequencies were not provided Authors report OSCEs and student scenarios exams showed participants had a good knowledge of the SOM role Educational impact: The training programme was effective in preparing SOM to undertake the final clinical assessments with student nurses
		Group discussion and reflection activities	Supporting failing students and those with additional needs
		Review of practice case scenarios	Provision of effective feedback
		Generation of portfolios with evidence of learning and mock action plans	Recourses to support SOMs
		Peer support group	Review of practice case scenarios
5. Bacon et al. (2015)	Population: Dietitians (n = 9) Format: Web-based self-directed learning, 7 h duration Context: Multiple contacts, online programme	Teaching and learning material delivered online	Learning outcomes and development of competency-based assessment skills Competency standard, assessment context and applying entry-level competency standards in clinical settings Simulated assessment practice to consolidate credible and defensible practice Comparison of grades with experienced supervisor	Rating of CEs' confidence levels assessing student competence pre- and post-training (1–10 numerical rating scale) Rating one video of a student's performance pre- and post-training Feedback from focus group/phone interviews, including how CEs ensure credible and defensible assessments Multiple-choice knowledge quiz post-training	Key findings: Pre-programme mean confidence rating 5.57 (range 2–9). No post-programme ratings available due to technical issues
		Provision of reading and video learning resources			Five participants (63%) rated student's performance as intermediate/beginner consistent with experienced supervisors. No post-programme ratings available due to technical issues
		Problem-based learning activities			Written feedback (n = 4/9, 44%) positive for (i) developing CEs' confidence to supervise students, (ii) utility of videos to understand performance and (iii) accessibility, utility and interdependence of teaching strategies
		Simulated assessment practice with 20 videos of students' performances available			Average score for quiz 86%
		Online peer discussion forum			Educational impact: Online education programme is a viable solution to develop clinical assessment skills and confidence. Allows access across a wide geographical community
6. Bacon et al. (2017)	Population: Dietitians (n = 26)	Didactic teaching of assessment concepts/processes	Review of recently revised assessment competencies Simulated assessment practice with opportunity to change rating and confidence scores after group discussion	Rating one video of a student's performance using VAS (1 novice −7 competent) and comparison of within-group ratings pre- and post-training	Key findings: No consistency in rating students' performances pre- or post-training. Pre-training median 4 (range 1–6) (n = 11/26), post-training median 4 (range 2–6) (n = 16/26) 35% (n = 9/26) of participants changed ratings to 4 after the training. No expert ratings were provided for video, therefore difficult to interpret the impact of these changes 31% (n = 8/26) increased assessment confidence, 27% (n = 7/26) reduced assessment confidence and 42% were unchanged (n = 11) post-training Qualitative themes identified (i) assessment discussion supports CEs' decisions, (ii) peer discussion during assessment leads to more holistic judgements and (iii) multiple sources of evidence/reflections are required for trustworthy assessment decisions Educational impact: Workplace assessment is a matter of consensus. Discussing judgements with colleagues can assist with a shared mental model of competence. The authors suggest that a programmatic assessment approach warrants further review for high-stakes workplace exams
	Format: In-person training, duration not specified	Workshop with simulated assessment practice using one video of a student's performance		Rating of CEs' confidence levels assessing a student's performance (numerical 1–10 rating scale)
	Context: Single training occasion, location not reported	Facilitated group discussion including sharing individual assessment grades and reflection on grades awarded		Global description of a student's overall performance
7. Moline et al. (2018)	Population: Radiation Therapists (n = 24)	Workshop with simulated assessment practice using one video of student's performances Facilitated group work	Simulated assessment practice using the standard evaluation tool Facilitated debriefing of grades awarded for simulated assessment practice and review of assessment criteria	Rating CEs' confidence levels assessing a student's performance using a Likert scale 1–5 (1 low - 5 high) pre- and post-training Rating CEs' confidence at 75 days post-training using the same scale (Likert 1–5)	Key findings: CEs' confidence rating a student's performance was significantly higher post-Workshop B compared to pre-workshop and at 75 days post- training (p < 0.05)
	Format: In-person training, 1 h duration				Educational impact
	Context: Single training occasion, tertiary clinical care centres				A short 1-h training workshop improves CEs' confidence to undertake competency assessments with the training effect persisting for at least 75 days
8. Alpine et al. (2021)	Population: Physiotherapists (n = 53)	Didactic teaching of assessment concepts	Review of assessment criteria and rating scale Introduction to FoRT principles Simulated assessment practice of ‘poor’ and ‘very good’ performances Facilitated group reflection and discussion of grades awarded for simulated assessment practice with an opportunity to change grades after group discussion Common pitfalls of assessment incorporated into group discussion	Rating videos of two different levels of students' performances pre- and post-group discussion against expert score Impact of demographics, professional experience, student supervision experience and professional grade on grading students	Key findings: Video 1 (poor performance): Alignment with expert ratings increased from 26% pre- discussion to 48% post discussion; 22% decreased scores from 4 (pass) to 3 (fail). For Video 2 (very good performance): Alignment with expert scores decreased from 43% pre-discussion to 41% post-discussion; 25% of participants lowered scores post discussion
	Format: In-person training, 1.5 h duration	Workshops based on Frame of Reference Training (FoRT) principles using simulated assessment practice with two videos of two different levels of performance			No significant difference based on gender and years qualified. Professional grade showed a difference for one learning outcome only. In terms of correlations, there was no significant relationship between supervision experience or the number of students supervised and change in scores
	Context: Single training occasion, University setting	Facilitated group discussion including sharing individual grades, CEs' rationale for same and reflection on grading			Educational impact: Significant changes in CEs' grades after group discussion, particularly for borderline pass/fail students, and improved alignment with expert ratings. Positive benefits of FoRT to assist decision-making in PBA.
9. Milne et al. (2022)	Population: Physiotherapists (n = 109)	Didactic teaching of assessment concepts/processes	Reflection/learning activities to guide understanding of entry-level practice thresholds Consideration of personal biases Simulated assessment practice of a student's performances Grade calibration through peer discussion of grades awarded during simulation assessment practice moderated by facilitators Calibration of assessment standards by identifying appropriate behaviours for each APP domain	Rating CEs' knowledge, skills, attributes (22 items) and confidence (six statements) using a five-point Likert scale (1 disagree – 5 strongly agree) Appraisal of CEs' perception of their ability to assess a student's performance using the APP.	Key findings: Post-training, statistically significant change in mean scores related to perceived knowledge, skills and attributes, relevant to student assessment, and overall confidence undertaking an assessment using APP (p < 0.001)
	Format: In-person training, 5 h duration	Workshop using simulated assessment practice with one video of a student's performance, followed by group discussion including rationale for ratings			Educational impact: A one-day in-person workshop significantly improved knowledge, skills, attributes and confidence to assess a student's performances
	Context: Single training occasion, University setting	Facilitated group discussion and reflection			The workshop assisted CEs' understanding of the assessment tool and competency standards

Note(s): Abbreviations: APP - Assessment of Physiotherapy Practice, CE - Clinical Educator, CIECP - Clinical Instructor Education and Credentialing Programme, CPI - Clinical Performance Instrument, COMPASS - Competency Assessment in Speech Pathology, FoRT - Frame of Reference Training, NMC - Nursing and Midwifery Council, OSCE - Objective Structured Clinical Examination, SOM - sign-off mentor, VAS - Visual Analogue Scale

[ViewLarge]

Sharing Unavailable