ANZCA & FPM CPD Program resources: Trainee assessment reviews

Abstract
Purpose: We aimed to develop a contemporary measure for anesthesia teaching and learning in the operating theatre that was applicable to a variety of training jurisdictions, the Measure for the Anaesthesia Theatre Educational Environment (MATE).

Methods: A systematic review of the literature and modified Delphi approach was used to identify items for content validity. Reliability and exploratory factor analyses were conducted after a pilot survey of trainees to show construct validity, with removal of redundant items. Item domains were identified through a global assessment of factor structure accuracy and relation to real-world constructs.

Results: Literature review generated an initial 73-item list. A modified Delphi approach with 24 experts identified 44 relevant items. The pilot survey generated 390 responses. Reliability analysis, exploratory factor analysis, and global assessment refined the measure to 33 items. Four domains were identified according to factor structure: teaching preparation and practice, assessment and feedback, procedures and responsibility, and overall atmosphere. The educational environment was rated by trainees at 74.6 ± 15.6% with excellent internal consistency (Cronbach's α = 0.975).

Conclusion: The MATE survey tool generated valid and reliable scores when measuring the educational environment in the operating theatre. Further research is required to investigate possible differences between the training countries and age of junior doctors and the associated underlying factors. Other researchers are invited to administer the survey and share results within a central database.

Abstract
The learning environment describes the way that trainees perceive the culture of their workplace. We audited the learning environment for trainees throughout Australia and New Zealand in the early stages of curriculum reform. A questionnaire was developed and sent electronically to a large random sample of Australian and New Zealand College of Anaesthetists trainees, with a 26% final response rate. This new instrument demonstrated good psychometric properties, with Cronbach's α ranging from 0.81 to 0.91 for each domain. The median score was equivalent to 78%, with the majority of trainees giving scores in the medium range. Introductory respondents scored their learning environment more highly than all other levels of respondents (P=0.001 for almost all comparisons). We present a simple questionnaire instrument that can be used to determine characteristics of the anaesthesia learning environment. The instrument can be used to help assess curricular change over time, alignment of the formal and informal curricula and strengths and weaknesses of individual departments.

Abstract
Background: Measurement of learning climates can serve as an indicator of a department's educational functioning.

Aim: This article describes the development and psychometric qualities of an instrument to measure learning climates in postgraduate specialist training: the Dutch Residency Educational Climate Test (D-RECT).

Method: A preliminary questionnaire was evaluated in a modified Delphi procedure. Simultaneously, all residents in the Netherlands were invited to fill out the preliminary questionnaire. We used exploratory factor analysis to analyze the outcomes and construct the definitive D-RECT. Confirmatory factor analysis tested the questionnaire's goodness of fit. Generalizability studies tested the number of residents needed for a reliable outcome.

Results: In two rounds, the Delphi panel reached consensus. In addition, 1278 residents representing 26 specialties completed the questionnaire. The Delphi panel's input in combination with the exploratory factor analysis of 600 completed surveys led to the definitive D-RECT, consisting of 50 items and 11 subscales (e.g., feedback, supervision, patient handover and professional relations between attendings). Confirmatory factor analyses of the remaining surveys confirmed the construct. The results showed that a feasible number of residents is needed for a reliable outcome.

Conclusion: D-RECT appears to be a valid, reliable and feasible tool to measure the quality of clinical learning climates.

Abstract
This paper describes the development and validation of a 40-item inventory, the Postgraduate Hospital Educational Environment Measure (PHEEM), by researchers in Scotland and the West Midlands using a combination of grounded theory and Delphi process. The instrument has since returned an alpha reliability >0.91 in two administrations in England and may be a useful instrument in the quality assurance process for postgraduate medical education and training.