Library resource guides: CPD for clinical support roles: I assess trainees, prevocational doctors & students in the workplace

Guide index

Key areas to consider in planning your CPD include:

Learning and growing as an assessor, including your professional supports and networks
Enhancing your communication skills
Planning effective teaching and learning
Facilitating learning in clinical settings, including by promoting a fair and culturally safe assessment environment
Making evidence-based judgements
Enhancing your understanding of assessment concepts and practices, including constructive feedback conversations
Enhancing your understanding of assessment fundamentals, workplace based assessment practice and how to ensure its quality
Understanding and managing own biases

Practice evaluation (category 1)

Undertake a critical reflection on your assessment work and develop a plan for practice change. Potential topics for reflection include your WBA performance/experiences (ideally based on student/trainee feedback) or integrating what you have learnt during WBA skills training into your practice.
Ask colleagues and trainees/students to provide you with multi-source feedback (clinical support), with a trusted colleague collating the feedback and discussing it with you.
Ask a colleague to complete a peer review of educational practice, focusing on the assessment of learning domain.
Survey trainees on the clinical learning environment in which training occurs (in your department, hospital, unit or practice) to identify areas for improvement. Plan and implement changes and remeasure the learning environment to evaluate the changes made. Claim this under quality improvement project. See the resources section for survey tool examples.

Knowledge & skills (category 2)

Undertake a formal educational qualification and claim time spent doing this under formal courses.
Read peer-reviewed journals relevant to workplace based assessment (see key areas to consider) Learning and growing as an assessor, including your professional supports and networks; Enhancing your communication skills; Planning effective teaching and learning; Facilitating learning in clinical settings, including by promoting a fair and culturally safe assessment environment Making evidence-based judgements; Enhancing your understanding of assessment concepts and practices, including constructive feedback conversations; Enhancing your understanding of assessment fundamentals, workplace based assessment practice and how to ensure its quality; Understanding and managing own biases and claim this as journal reading.
Attend face-to-face or virtual learning sessions relevant to your WBA assessor role (for example: conferences, podcasts, webinars; see key areas to considerLearning and growing as an assessor, including your professional supports and networks; Enhancing your communication skills; Planning effective teaching and learning; Facilitating learning in clinical settings, including by promoting a fair and culturally safe assessment environment Making evidence-based judgements; Enhancing your understanding of assessment concepts and practices, including constructive feedback conversations; Enhancing your understanding of assessment fundamentals, workplace based assessment practice and how to ensure its quality; Understanding and managing own biases). Claim time spent doing this under learning sessions.
Assess trainee/student performance in the workplace using a structured form with constructive performance feedback. Claim time spent doing this under WBA of trainees.

The college welcomes suggestions on additional resources and topics for CPD on assessing trainees, prevocational doctors & students.

Learn@ANZCA

Learn@ANZCA (formerly Networks) is the college's learning management system.

Note: Some resources located in Learn@ANZCA require that you first register before accessing.

ANZCA: Setting up SSO and MFA Everything you need to know about single-sign on (SSO), multi-factor authentication (MFA), and how we're keeping your data safe.

Learn@ANZCA resources

ANZCA Educators Program [Learn@ANZCA]
The AEP has been developed as a practical teaching course to support all clinicians who have a teaching role and particularly those who work with trainees, including supervisors, fellows and senior trainees. The course will prepare you with the appropriate knowledge and skills in educational principles to facilitate learning and teach with results in the anaesthetic and pain medicine environment.
Fundamentals of feedback [Learn@ANZCA]
Comprised of seven e-learning modules - including an introductory module - designed to up-skill and support you to have effective feedback conversations in all aspects of your daily practice, the Fundamentals of Feedback course is designed to introduce you to a conversational framework that may be applied to feedback conversations, both clinical and simulated settings.
WBA Support Resources [Learn@ANZCA]
The WBA Support Resources site is a key source of all information relating to WBAs, including workshops, webinars, practical tips, checklists and a comprehensive list of frequently asked questions. Specific resources are available for supervisors of training, WBA assessors and trainees to help everyone understand the philosophy of WBAs and how to use them to enhance learning.

CPD Note: CPD participants will automatically have this activity populated in the CPD portal.

ANZCA & FPM courses

ANZCA Educators Program (AEP)
This is an interactive five module course, delivered over 1.5 days, designed to equip anaesthetists and faculty of pain medicine specialists with the knowledge and skills to be competent clinical teachers.

ANZCA & FPM resources

CPD Activity Guide
This guide contains a list of all the activities associated with the ANZCA and FPM CPD Program, as well as links to associated individual activity guides and forms.

Educator Competency Framework by Australian and New Zealand College of Anaesthetists The ANZCA Educator Competency Framework was developed after an analysis of educator roles and responsibilities in specialty training, a review and thematic analysis of published competency domains and frameworks, a survey of educators, expert group discussion, and feedback from stakeholders. Its broad design covers all the competencies required of educators for specialty training, from the basic competencies expected of all clinical specialists to the more advanced competencies required of oversight education committee members. It is designed to be used by any institution involved in medical specialty training to map the competencies required of different educator roles.
Publication Date: August 2022

ANZCA: Medical Education Library Guide
This guide has been designed for medical personnel and educators interested in medical education to locate relevant resources on this topic, including those available through the ANZCA library.

External resources

Australian Medical Council (AMC) Limited: AMC Assessment Website
Designed for all stakeholders involved in assessment of learners across the continuum of medical education, this website houses resources and information to share learning and support improvement in assessment
Harvard: Project Implicit - Harvard implicit association tests
The mission of Project Implicit is to educate the public about bias and to provide a “virtual laboratory” for collecting data on the internet.

Tools to Audit Learning Environment

Anaesthesia-specific tools

MATE (Measure for the Anaesthesia Theatre educational Environment) for teaching in the OR:

Sidhu NS, Clissold E. Developing and validating a tool for measuring the educational environment in clinical anesthesia. Can J Anaesth. 2018;65(11):1228-1239.
more...less...
Abstract
Purpose: We aimed to develop a contemporary measure for anesthesia teaching and learning in the operating theatre that was applicable to a variety of training jurisdictions, the Measure for the Anaesthesia Theatre Educational Environment (MATE).

Methods: A systematic review of the literature and modified Delphi approach was used to identify items for content validity. Reliability and exploratory factor analyses were conducted after a pilot survey of trainees to show construct validity, with removal of redundant items. Item domains were identified through a global assessment of factor structure accuracy and relation to real-world constructs.

Results: Literature review generated an initial 73-item list. A modified Delphi approach with 24 experts identified 44 relevant items. The pilot survey generated 390 responses. Reliability analysis, exploratory factor analysis, and global assessment refined the measure to 33 items. Four domains were identified according to factor structure: teaching preparation and practice, assessment and feedback, procedures and responsibility, and overall atmosphere. The educational environment was rated by trainees at 74.6 ± 15.6% with excellent internal consistency (Cronbach's α = 0.975).

Conclusion: The MATE survey tool generated valid and reliable scores when measuring the educational environment in the operating theatre. Further research is required to investigate possible differences between the training countries and age of junior doctors and the associated underlying factors. Other researchers are invited to administer the survey and share results within a central database.

ACLEI (Anaesthesia Clinical Learning Environment Instrument) for overall anaesthesia learning environment:

Smith NA, Castanelli DJ. Measuring the clinical learning environment in anaesthesia. Anaesth Intensive Care. 2015;43(2):199-203. Keywords: anaesthesia; curriculum; learning environment; psychometrics; questionnaires; workplace.
more...less...
Abstract
The learning environment describes the way that trainees perceive the culture of their workplace. We audited the learning environment for trainees throughout Australia and New Zealand in the early stages of curriculum reform. A questionnaire was developed and sent electronically to a large random sample of Australian and New Zealand College of Anaesthetists trainees, with a 26% final response rate. This new instrument demonstrated good psychometric properties, with Cronbach's α ranging from 0.81 to 0.91 for each domain. The median score was equivalent to 78%, with the majority of trainees giving scores in the medium range. Introductory respondents scored their learning environment more highly than all other levels of respondents (P=0.001 for almost all comparisons). We present a simple questionnaire instrument that can be used to determine characteristics of the anaesthesia learning environment. The instrument can be used to help assess curricular change over time, alignment of the formal and informal curricula and strengths and weaknesses of individual departments.

Pain medicine- specific tools

Dutch Residency Educational Climate Test (D-RECT):

Boor K, Van Der Vleuten C, Teunissen P, Scherpbier A, Scheele F. Development and analysis of D-RECT, an instrument measuring residents' learning climate. Med Teach. 2011;33(10):820-827.
more...less...
Abstract
Background: Measurement of learning climates can serve as an indicator of a department's educational functioning.

Aim: This article describes the development and psychometric qualities of an instrument to measure learning climates in postgraduate specialist training: the Dutch Residency Educational Climate Test (D-RECT).

Method: A preliminary questionnaire was evaluated in a modified Delphi procedure. Simultaneously, all residents in the Netherlands were invited to fill out the preliminary questionnaire. We used exploratory factor analysis to analyze the outcomes and construct the definitive D-RECT. Confirmatory factor analysis tested the questionnaire's goodness of fit. Generalizability studies tested the number of residents needed for a reliable outcome.

Results: In two rounds, the Delphi panel reached consensus. In addition, 1278 residents representing 26 specialties completed the questionnaire. The Delphi panel's input in combination with the exploratory factor analysis of 600 completed surveys led to the definitive D-RECT, consisting of 50 items and 11 subscales (e.g., feedback, supervision, patient handover and professional relations between attendings). Confirmatory factor analyses of the remaining surveys confirmed the construct. The results showed that a feasible number of residents is needed for a reliable outcome.

Conclusion: D-RECT appears to be a valid, reliable and feasible tool to measure the quality of clinical learning climates.

Post-graduate Hospital Educational Environment Measure (PHEEM):

Roff S, McAleer S, Skinner A. Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach. 2005;27(4):326-331.
more...less...
Abstract
This paper describes the development and validation of a 40-item inventory, the Postgraduate Hospital Educational Environment Measure (PHEEM), by researchers in Scotland and the West Midlands using a combination of grounded theory and Delphi process. The instrument has since returned an alpha reliability >0.91 in two administrations in England and may be a useful instrument in the quality assurance process for postgraduate medical education and training.