Library resource guides: ANZCA and FPM CPD Program resources: Practice audit (clinical support)

Guide index

Practice audit (clinical support)

Collecting data on practice in areas that don’t involve direct patient care, reflecting on the results and considering practice changes if areas for improvement are identified.

***

Relevant to those with clinical support roles or those who practice without direct patient care
May involve one CPD participant or a group in single or multiple disciplines.
May focus on individual, team, departmental, group, hospital or health service outcomes.

Undertake a comparison of safety and quality datasets with best practice standards in that domain.
Undertake an audit of the outcomes of a department, group, hospital or health service, compared with state, national or international benchmarks.
Undertake an audit of own performance in an area of practice compared with those of peers (department or group of colleagues).
Undertake an audit of supervision and teaching in your hospital, department or practice
Undertake a gender equity survey using the Gender equity survey toolkit (see ANZCA & FPM resources section)
Undertake an environmental sustainability audit using the Environmental sustainability audit tool (see ANZCA & FPM resources section)

See the Practice audit (clinical support) guideline for further information concerning this activity.

See also: The CPD for clinical support roles guide for comprehensive support for those with clinical support roles or those who practice without direct patient care

Following the practice audit (clinical support), you may choose to complete a report of audit findings
If the audit results are discussed with a colleague, the colleague can record this as a review of ANZCA and FPM fellows activity.
Following the practice audit (clinical support), you may choose to undertake an individual critical reflection activity n what went well/not so well or what would you do differently in a future audit?

CPD Value: CPD participants may claim this under the Practice evaluation - measuring outcomes practice audit (clinical support) activity.

External resources

The Royal College of Anaesthetists (RCoA) has developed a publication titled Raising the Standards: RCoA quality improvement compendium, which provides examples of achievable audits focused mainly on measurement against defined process standards.

ANZCA gratefully acknowledges the RCoA’s willingness to make this document available for our participants. Ready-to-use audit topics and samples relevant to clinical care are presented as a series of chapters (which include perioperative care, pain medicine, intensive care medicine).

Raising the Standards: RCoA Quality Improvement Compendium, 4e by Chereshneva M, Johnston C, Colvin JR, Peden CJ [eds]. The Quality Improvement Compendium, previously known as the Audit Recipe Book, provides a manual of quality improvement (QI) and audit tools for anaesthetists.
ISBN: 9781527265226
Publication Date: September 2020
Chapter 10 - Pain Medicine, Raising the Standards: RCoA Quality Improvement Compendium, 4e by Dr Matthew Brown and Dr Manohar Lal Sharm [eds] The Quality Improvement Compendium, previously known as the Audit Recipe Book, provides a manual of quality improvement (QI) and audit tools for anaesthetists.
ISBN: 9781527265226
Publication Date: September 2020

The Royal Australasian College of Medical Administrators (RACMA) in their “Continuing Professional Development Handbook” include guidance on auditing outcomes of own practice or that of your organisation.3 They outline steps for audits and provide suggestions about audits and relevant standards and indicators.

ANZCA gratefully acknowledges the RACMA’s willingness to make this document available for our participants.

Continuing Professional Development > Handbook RACMA Member 2023 by Royal Australasian College of Medical Administrators (RACMA) This handbook describes RACMA's CPD program and provides examples of the types of activities that we believe will help you in your professional development. The range of activities that are suitable for CPD is very wide and the suggestions in this handbook are not exhaustive. We encourage you to design and undertake the activities which will be of most benefit for your particular professional development needs.
Publication Date: 2023

The Medical Council of New Zealand website includes some examples of audits relevant to clinical support or practice without direct patient care.

MCNZ: Audit of medical practice
An audit of medical practice is a systematic, critical analysis of the quality of a doctor’s own practice, the results of which are used to improve clinical care and/or health outcomes, or to confirm that current management is consistent with the current available evidence or accepted consensus guidelines.

A Best Practice Guide for Continuous Practice Improvement by New Zealand Council of Medical Colleges, Te Kaunihera o Ngā Kāreti Rata o Aotearoa. This Guide is a framework for use when developing or reviewing programmes set up to demonstrate the competence and performance of medical specialists. The Best Practice Guide was produced as a joint project between the Council of Medical Colleges (CMC) and Member Colleges, the Medical Council of New Zealand (MCNZ), the Ministry of Health (MoH) and District Health Board (DHB) Chief Medical Officers.
Publication Date: 1 February 2016
Best Practice in Clinical Audit by Healthcare Quality Improvement Partnership This document is the result of wide consultation and workshops with clinicians, service managers, and clinical audit staff, as well as representatives of a range of professional bodies including the Academy of Medical Royal Colleges.
ISBN: 9781907561498
Publication Date: April 2020
A Guide to Using Data for Health Care Quality Improvement by Victorian Government Department of Human Services
Publication Date: June 2008
The How-to Guide for Measurement for Improvement by Clarke J, Davidge M, James L. NHS Institute for Innovation and Improvement This guide is designed to help you to this in your improvement projects. It is in two parts. Part 1 explains what measurement for improvement is and how it differs from other sorts of measurement that you will have come across. Part 2 talks you through the process of collecting, analysing and reviewing data. If you are familiar with the Model for Improvement and how to use it, you can skip Part 1 and go straight to Part 2.
Publication Date: 2009
Non Clinical Work and Revalidation: report and recommendations from the Non Clinical Work Group of the Academy of Medical Royal Colleges by Academy of Medical Royal Colleges
Publication Date: 2009
Public Reporting of Safety and Quality in Public and Private Hospitals: Literature Review and Environment Scan by Australian Commission on Safety and Quality in Health Care This report provides a summary of findings from a literature review and environment scan undertaken to inform options for national public reporting standards of safety and quality in health care across public and private hospitals in Australia.
ISBN: 9781925665802
Publication Date: March 2019

AANA: Practice Management
Practice management tools and insights help you confidently manage practice events, practice resources, anesthesia staffing, and patient care needs.
HQIP: Best Practice in Clinical Audit
The purpose of this document is to set out updated criteria for best practice in local clinical audit
QLD Health: Audit tools for National Safety and Quality Health Service Standards
A range of audit tools have been developed for facilities to use to collect data and evidence of meeting the standards.

Backhouse A, Ogunlayi F. Quality improvement into practice. BMJ. 2020;368:m865. Published 2020 Mar 31.
Limb C, Fowler A, Gundogan B, Koshy K, Agha R. How to conduct a clinical audit and quality improvement project. Int J Surg Oncol (N Y). 2017;2(6):e24. Keywords: Audit; Clinical governance; Design; Leadership and management; Project design; QIP; Quality improvement project; Service improvement.
more...less...
Abstract
Audits and quality improvement projects are vital aspects of clinical governance and continual service improvement in medicine. In this article we describe the process of clinical audit and quality improvement project. Guidance is also provided on how to design an effective audit and bypass barriers encountered during the process.

Tools to audit the clinical learning environment

Anaesthesia-specific tools

MATE (Measure for the Anaesthesia Theatre educational Environment) for teaching in the OR:

Sidhu NS, Clissold E. Developing and validating a tool for measuring the educational environment in clinical anesthesia. Can J Anaesth. 2018;65(11):1228-1239.
more...less...
Abstract
Purpose: We aimed to develop a contemporary measure for anesthesia teaching and learning in the operating theatre that was applicable to a variety of training jurisdictions, the Measure for the Anaesthesia Theatre Educational Environment (MATE).

Methods: A systematic review of the literature and modified Delphi approach was used to identify items for content validity. Reliability and exploratory factor analyses were conducted after a pilot survey of trainees to show construct validity, with removal of redundant items. Item domains were identified through a global assessment of factor structure accuracy and relation to real-world constructs.

Results: Literature review generated an initial 73-item list. A modified Delphi approach with 24 experts identified 44 relevant items. The pilot survey generated 390 responses. Reliability analysis, exploratory factor analysis, and global assessment refined the measure to 33 items. Four domains were identified according to factor structure: teaching preparation and practice, assessment and feedback, procedures and responsibility, and overall atmosphere. The educational environment was rated by trainees at 74.6 ± 15.6% with excellent internal consistency (Cronbach's α = 0.975).

Conclusion: The MATE survey tool generated valid and reliable scores when measuring the educational environment in the operating theatre. Further research is required to investigate possible differences between the training countries and age of junior doctors and the associated underlying factors. Other researchers are invited to administer the survey and share results within a central database.

ACLEI (Anaesthesia Clinical Learning Environment Instrument) for overall anaesthesia learning environment:

Smith NA, Castanelli DJ. Measuring the clinical learning environment in anaesthesia. Anaesth Intensive Care. 2015;43(2):199-203. Keywords: anaesthesia; curriculum; learning environment; psychometrics; questionnaires; workplace.
more...less...
Abstract
The learning environment describes the way that trainees perceive the culture of their workplace. We audited the learning environment for trainees throughout Australia and New Zealand in the early stages of curriculum reform. A questionnaire was developed and sent electronically to a large random sample of Australian and New Zealand College of Anaesthetists trainees, with a 26% final response rate. This new instrument demonstrated good psychometric properties, with Cronbach's α ranging from 0.81 to 0.91 for each domain. The median score was equivalent to 78%, with the majority of trainees giving scores in the medium range. Introductory respondents scored their learning environment more highly than all other levels of respondents (P=0.001 for almost all comparisons). We present a simple questionnaire instrument that can be used to determine characteristics of the anaesthesia learning environment. The instrument can be used to help assess curricular change over time, alignment of the formal and informal curricula and strengths and weaknesses of individual departments.

Pain medicine- specific tools

Dutch Residency Educational Climate Test (D-RECT):

Boor K, Van Der Vleuten C, Teunissen P, Scherpbier A, Scheele F. Development and analysis of D-RECT, an instrument measuring residents' learning climate. Med Teach. 2011;33(10):820-827.
more...less...
Abstract
Background: Measurement of learning climates can serve as an indicator of a department's educational functioning.

Aim: This article describes the development and psychometric qualities of an instrument to measure learning climates in postgraduate specialist training: the Dutch Residency Educational Climate Test (D-RECT).

Method: A preliminary questionnaire was evaluated in a modified Delphi procedure. Simultaneously, all residents in the Netherlands were invited to fill out the preliminary questionnaire. We used exploratory factor analysis to analyze the outcomes and construct the definitive D-RECT. Confirmatory factor analysis tested the questionnaire's goodness of fit. Generalizability studies tested the number of residents needed for a reliable outcome.

Results: In two rounds, the Delphi panel reached consensus. In addition, 1278 residents representing 26 specialties completed the questionnaire. The Delphi panel's input in combination with the exploratory factor analysis of 600 completed surveys led to the definitive D-RECT, consisting of 50 items and 11 subscales (e.g., feedback, supervision, patient handover and professional relations between attendings). Confirmatory factor analyses of the remaining surveys confirmed the construct. The results showed that a feasible number of residents is needed for a reliable outcome.

Conclusion: D-RECT appears to be a valid, reliable and feasible tool to measure the quality of clinical learning climates.

Post-graduate Hospital Educational Environment Measure (PHEEM):

Roff S, McAleer S, Skinner A. Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach. 2005;27(4):326-331.
more...less...
Abstract
This paper describes the development and validation of a 40-item inventory, the Postgraduate Hospital Educational Environment Measure (PHEEM), by researchers in Scotland and the West Midlands using a combination of grounded theory and Delphi process. The instrument has since returned an alpha reliability >0.91 in two administrations in England and may be a useful instrument in the quality assurance process for postgraduate medical education and training.

Learn@ANZCA

Use the links below to access additional support resources contained on the Learn@ANZCA platform.

Note: Resources located in Learn@ANZCA require that you first register before accessing.

Clinical Audit Samples [Learn@ANZCA]
The practice evaluation component of the CPD program includes the activity 'Clinical audit of own practice or significant input into a group of audit practice'. This learn module contains the large number of templates available.

ANZCA & FPM resources

CPD Activity Guide
This guide contains a list of all the activities associated with the ANZCA and FPM CPD Program, as well as links to associated individual activity guides and forms.
ANZCA and FPM CPD Program: Practice audit (clinical support) guideline
This guideline assists CPD participants complete the Practice audit (clinical support) (PA-CS) activity.
ANZCA and FPM CPD Program: Practice audit (clinical support) CPD verification form

ANZCA: CPD for Clinical Support Roles Library Guide
The purpose of this resources guide is to assist fellows to match CPD activities with the nature of the work roles that they undertake. The target audience is those with clinical support roles alongside their work with patients and those who practice without direct patient care.

Gender Equity Survey Toolkit

Gender equity survey toolkit
In 2022, ANZCA performed its inaugural Gender equity survey and asked the question - “Does gender matter in a career in anaesthesia?”. The results of this survey showed that despite increasing numbers of female anaesthetists, gender inequity remains an issue affecting anaesthetics across Australia and New Zealand. In conducting the 2022 survey only 1200 ANZCA fellows could be invited to participate in the survey due to ANZCA Clinical Trials Network (CTN) guidelines. These participants were randomly selected from the ANZCA database, and the GESC received feedback during the survey period that many people wished they had been selected to take part.

This toolkit contains six documents to assist you in performing a local gender equity survey:

CPD Value: CPD participants may claim this under the Practice evaluation - measuring performance practice audit (clinical support) activity.

Environmental Sustainability Audit Tool

Environmental Sustainability Audit Tool by Australian and New Zealand College of Anaesthetists
Publication Date: 2021
This document has been developed by the ANZCA Environmental Sustainability Working Group to assist departments and practitioners to develop and maintain practices that promote environmental sustainability in line with ANZCA’s professional document PS64 Statement on Environmental Sustainability in Anaesthesia and Pain Medicine Practice and the ANZCA Statement on Climate Change.