Perioperative medicine training resources: U2M18: Research outcome measures and data registries

Abstract
Background: Clinical quality registries (CQRs) are playing an increasingly important role in improving health outcomes and reducing health care costs. CQRs are established with the purpose of monitoring quality of care, providing feedback, benchmarking performance, describing pattern of treatment, reducing variation and as a tool for conducting research.

Objectives: To synthesise the impact of clinical quality registries (CQRs) as an 'intervention' on (I) mortality/survival; (II) measures of outcome that reflect a process or outcome of health care; (III) health care utilisation; and (IV) healthcare-related costs.

Methods: The following electronic databases were searched: MEDLINE, EMBASE, CENTRAL, CINAHL and Google Scholar. In addition, a review of the grey literature and a reference check of citations and reference lists within articles was undertaken to identify relevant studies in English covering the period January 1980 to December 2016. The PRISMA-P methodology, checklist and standard search strategy using pre-defined inclusion and exclusion criteria and structured data extraction tools were used. Data on study design and methods, participant characteristics attributes of included registries and impact of the registry on outcome measures and/or processes of care were extracted.

Results: We identified 30102 abstracts from which 75 full text articles were assessed and finally 17 articles were selected for synthesis. Out of 17 studies, six focused on diabetes care, two on cardiac diseases, two on lung diseases and others on organ transplantations, rheumatoid arthritis, ulcer healing, surgical complications and kidney disease. The majority of studies were "before after" design (#11) followed by cohort design (#2), randomised controlled trial (#2), experimental non randomised study and one cross sectional comparison. The measures of impact of registries were multifarious and included change in processes of care, quality of care, treatment outcomes, adherence to guidelines and survival. Sixteen of 17 studies demonstrated positive findings in their outcomes after implementation of the registry.

Conclusions: Despite the large number of published articles using data derived from CQRs, few have rigorously evaluated the impact of the registry as an intervention on improving health outcomes. Those that have evaluated this impact have mostly found a positive impact on healthcare processes and outcomes.

Abstract
Accurate information regarding prognosis is fundamental to optimal clinical care. The best approach to assess patient prognosis relies on prediction models that simultaneously consider a number of prognostic factors and provide an estimate of patients' absolute risk of an event. Such prediction models should be characterized by adequately discriminating between patients who will have an event and those who will not and by adequate calibration ensuring accurate prediction of absolute risk. This Users' Guide will help clinicians understand the available metrics for assessing discrimination, calibration, and the relative performance of different prediction models. This article complements existing Users' Guides that address the development and validation of prediction models. Together, these guides will help clinicians to make optimal use of existing prediction models.

Abstract
Identifying and acting on variations from good practice is one of the critical tasks of clinical governance. We describe one aspect of Queensland's post-Bundaberg clinical governance arrangements: the use of variable life-adjusted displays (VLADs) to monitor outcomes of care in the 87 largest public and private hospitals in Queensland, which together account for 83% of all hospital activity. VLAD control charts were created for 31 clinical indicators using routinely collected data, and are disseminated monthly. About a third of hospitals had a run of cases in the 3-year period that flagged at the 30% level (local level investigation). For three indicators, about one in five hospitals had sufficiently cumulatively more deaths than statistically expected that the hospital was highlighted for state-wide review. VLADs do not provide definitive answers about the quality of care. They are used to develop ideas about why variations in reported outcomes occur and suggest possible solutions, be they ways of improving data quality, improving casemix adjustment, or implementing system changes to improve quality of care. Critical to the approach is that there is not just monitoring - the monitoring is tied in with systems that ensure that investigation, learning and action occur as a result of a flag.

Abstract
This paper reviews the field of outcomes measurement in anesthesia and surgery, emphasizing those outcomes that may be influenced by perioperative care. Data sources for outcomes measurement are described, and the concept of risk adjustment is introduced. The basic mechanics of outcomes measurement and its pitfalls are explained. Finally, specific perioperative outcomes - mortality, readmission and composite outcomes - are described and their limitations are considered.

Abstract
The history of monitoring the outcomes of health care by external agencies can be traced to ancient times. However, the danger, now as then, is that in the search for improvement, comparative measures of mortality and morbidity are often overinterpreted, resulting in judgments about the underlying quality of care. Such judgments can translate into performance management strategies in the form of capricious sanctions (such as star ratings) and unjustified rewards (such as special freedoms or financial allocations). The resulting risk of stigmatising an entire institution injects huge tensions into health-care organisations and can divert attention from genuine improvement towards superficial improvement or even gaming behaviour (ie, manipulating the system). These dangers apply particularly to measures of outcome and throughput. We argue that comparative outcome data (league tables) should not be used by external agents to make judgments about quality of hospital care. Although they might provide a reasonable measure of quality in some high-risk surgical situations, they have little validity in acute medical settings. Their use to support a system of reward and punishment is unfair and, unsurprisingly, often resisted by clinicians and managers. We argue further that although outcome data are useful for research and monitoring trends within an organisation, those who wish to improve care for patients and not penalise doctors and managers, should concentrate on direct measurement of adherence to clinical and managerial standards.

Abstract
A pyramid has expressed the idea of hierarchy of medical evidence for so long, that not all evidence is the same. Systematic reviews and meta-analyses have been placed at the top of this pyramid for several good reasons. However, there are several counterarguments to this placement. We suggest another way of looking at the evidence-based medicine pyramid and explain how systematic reviews and meta-analyses are tools for consuming evidence—that is, appraising, synthesising and applying evidence.

Abstract
Medical registries provide highly reliable data, challenged hierarchically only by randomized controlled trials. Although registries have been used in several fields of medicine for more than a century and a half, their key role is frequently overlooked and poorly recognized. Medical registries have evolved from calculating basic epidemiological data (incidence, prevalence, mortality) to diverse applications in disease prevention, early diagnosis and screening programs, treatment response, health care planning, decision making and disease control programs. Implementing, maintaining and running a medical registry requires substantial effort. Developing the registry represents a complex task and is one of the major barriers in widespread use of registries. Medical registries have potential to evolve to a next generation by taking benefit from recent semantic web technology developments. This paper is aimed at providing a summary of the basic information available on medical registries and to highlight the progress and potential applications in this field.

Abstract
Background: The reliability of meta-analysis (MA) in predicting the findings of subsequent large randomized controlled trials (RCTs) has not been assessed in perioperative medicine and anaesthesia.

Methods: Using Medline and PubMed, large RCTs (n≥1000) published since 2000 in the anaesthesia and perioperative medicine/critical care literature were identified. All previous MAs of RCTs investigating the same intervention and population were sourced. For all reported major morbid endpoints common to each, results (significant/non-significant P<0.05) were compared.

Results: 18 large RCTs and 44 prior MAs investigating the effects of 16 interventions were identified. Where endpoint results in the large RCTs were each compared with the single largest recent preceding MA, 35 of a total of 57 outcomes were predicted correctly by the MAs (61.4%). The odds ratio for a significant result from MA compared with the subsequent large RCT was 3.6, P=0.033 Bonferroni corrected. The positive predictive value of MAs was 22.7%; the negative predictive value was 85.7%, Kappa was 0.094 indicating slight agreement. The estimated power for each endpoint for large RCTs and MAs were similar, but the median study size for large RCTs was larger than that of the MAs, n=4,482 vs 1,389, P<0.0001.

Conclusions: There was a strong tendency towards positive findings in MA not substantiated by subsequent large RCTs, which was not attributable to differences in study power. This finding suggests caution in clinical decision-making in anaesthesia and perioperative medicine based on findings of meta-analysis.