Identification of risk factors by systematic review and development of risk-adjusted models for surgical site infection

Article Type: Health technology assessment From: Clinical Governance: An International Journal, Volume 17, Issue 1

C. Gibbons, J. Bruce, J. Carpenter, A.P. Wilson, J. Wilson, A. Pearson, D.L. Lamping, Z.H. Krukowski and B.C. Reeves

Background to the research

Surgical site infections (SSIs) are complications of surgery that cause significant postoperative morbidity. They are costly to health services and inconvenient, painful and potentially fatal to affected patients. Rates of SSI have been observed to vary widely by hospital and may be influenced by surgical management and other aspects of the quality of health care. SSI rate (SSI per cent) has been proposed as a potential indicator of the quality of care in the context of clinical governance and monitoring of the performance of NHS organisations against targets.

The risk of developing an SSI is likely to be influenced by the characteristics of patients, of operations and postoperative care. Therefore,the use of SSI as a performance indicator requires hospital-specific rates to be risk adjusted. This research sought to identify important risk factors for SSI in defined contexts, whether surgery specific or generic, and investigate the feasibility of risk-adjusting SSI per cent.

Aim and objectives of the research

The aim of the proposed research was to investigate methods for the risk adjustment of rates of SSI. We proposed to address the following specific objectives:

• To identify risk factors for SSI, criteria for the stratification of surgical procedures and evidence about the importance of postdischarge surveillance (PDS) from systematic reviews of the literature.

• To test whether or not ‘short-listed’ variables from the literature are risk factors in available SSI surveillance databases. To identify in univariable analyses other potential risk factors from available databases and to investigate interactions between risk factors.

• To develop models for making risk-adjusted comparisons between hospitals.

• To investigate modifications of the definition of SSI used by the Centers for Disease Control and Prevention (CDC) and the impact of modified definitions on the importance (use for prediction) of risk factors identified.

How the research was conducted

Reviews of the literature

Four systematic reviews of the literature were carried out. These reviews sought to identify:

• 1.

  surgery-specific risk factors for SSI following joint replacement;

• 2.

  surgery-specific risk factors for SSI following large bowel surgery;

• 3.

  generic risk factors (relevant to many surgical procedures) for SSI that are not included in existing SSI risk indices; and

• 4.

  risk factors for SSIs detected by following up patients after discharge.

Systematic searches were conducted on two biomedical databases, MEDLINE and EMBASE (1966–2004 and 1980–2004, respectively). Search strategies consisted of medical subject headings and free-text terms relating to surgical infection (surgical wound infection/SSI/postoperative infection), risk adjustment (risk assessment/factor/adjustment/stratification/modelling) and,where appropriate, the surgical area being reviewed or terms describing PDS. The review also used literature identified by a previous systematic review.

Agreement between definitions of surgical site infection

This part of the research used data collected by SSI surveillance of cardiac,thoracic, orthopaedic, general, obstetric, gynaecological, urological,maxillofacial, plastic and vascular surgical specialties in one UK hospital. The data, for 5,804 surgical wounds in 4,773 patients, allowed four SSI definitions to be applied:

• 1.

  ASEPSIS (Additional treatment, the presence of Serous discharge, Erythema,Purulent exudate, and Separation of the deep tissues, the Isolation of bacteria and the duration of inpatient Stay);

• 2.

  1992 definition of the CDC;

• 3.

  a modified version of the 1992 CDC definition used for SSI surveillance in England; and

• 4.

  a definition based on pus.

Patients were contacted by post or telephone 1–2 months after their operations to complete a PDS questionnaire designed to detect SSIs arising after discharge from hospital. SSIs identified by different definitions were tabulated and agreement between definitions was quantified.

Validation of definitions of surgical site infection

This part of the research used an updated version of the above data set from the same UK hospital, describing 11,124 wounds in 8691 patients. We constructed a set of clinical outcomes that wound infection would be expected to influence or cause:

• clinical actions that were likely to reflect both mild (prescription of antibiotic) and severe infection (wound retreated);

• patients’ views about whether or not there was a problem with the healing of their wounds; and

• length of hospital stay, reflecting health service resource use.

Modifications were made to SSI definitions to try to ensure that they were independent of the outcomes. We then developed logistic regression models to quantify the ability of alternative SSI definitions to predict the outcomes.

Surgical site infection risk modelling

This part of the research used data submitted to the UK Surgical Site Infection Surveillance Service (at the time, the Nosocomial Infection National Surveillance Scheme (NINSS)). Hospitals taking part carried out surveillance of one or more of 12 categories of surgical procedure, e.g. large bowel surgery,coronary artery bypass graft (CABG) or hip replacement. In order for data to be included, there was a requirement for a hospital to carry out surveillance for at least three consecutive months. Hospitals submitted data about key risk factors for SSI (including the National Nosocomial Infections Surveillance(NNIS) risk index, demographic information about patients, and characteristics of the operation and wound) and information about SSIs that developed during the hospital stay. Univariable logistic regression analyses were initially carried out on the entire data set of 113,824 operations, stratifying by surgical procedure and then for each procedure separately. Multivariable risk models,with hospitals fitted as random effects, were then developed for each procedure. Effect modification of risk factors by hospital was investigated in multilevel models.

Research findings

Reviews of the literature

The literature reviewed was found to be mainly of poor methodological quality, preventing quantitative summaries of the risk conferred by specific risk factors. The reviews of surgery-specific risk factors, other than those that make up the established risk indices, identified other factors associated with increasing risk of SSI. This has also been suggested for operations other than those that we reviewed. Some risk factors are unequivocally surgery specific, but others may apply to a range of procedures. The factor most commonly identified by the review of generic risk factors was duration of preoperative stay in hospital. The review of SSIs detected by PDS demonstrated that a significant proportion of SSIs develop after discharge and that the need to include PDS is an important consideration for procedures where the length of hospital stay is short or likely to vary over time or between institutions.

Agreement between definitions of surgical site infection

There was wide variation in the frequency of SSI identified using different definitions. Using existing CDC and ASEPSIS definitions of SSI (most and least sensitive definitions), over twice as many wounds were classified as infected by one definition only as were classified as infected by both. Different SSI definitions also classified different wounds as being infected, although some wounds were classified as infected by all definitions.

Validation of definitions of surgical site infection

Both ASEPSIS and CDC SSI definitions had a broadly similar ability to predict the chosen clinical outcomes; areas under receiver operating characteristic curves ranged from 0.75 to 0.88, except for prediction of prolonged hospital stay (0.64). These findings are paradoxical given the poor agreement between definitions in classifying individual wounds. There may be elements of each definition that are important in identifying the outcomes but which are not common to both, or the ability to predict the outcomes may depend on only a subset of features that are common to both. These possibilities suggest that there is an opportunity to produce a better definition by combining the elements from different definitions or by dropping redundant ones.

Surgical site infection risk modelling

Univariable models highlighted that components of existing risk indices should be modelled separately and that there was effect modification of risk factors by surgical procedure. The risk factors included in best-fit multivariable models varied by surgical procedure, as did the effects of risk factors included in the models. This conclusion applies to components of existing risk indices as well as to other factors considered in the analyses. Of the components in established risk indices, operative duration appeared to be an important risk factor for all operations, except for hip replacement. Wound class was included least often because some wound classes were not applicable to some surgical procedures or were combined because of small numbers. The American Association of Anesthesiologists class was a consistent risk factor for most surgery categories (except open reduction of fractures); its effect was uncertain for limb amputation and vascular surgery because of the small sample sizes available.

Age and gender were included in all models. The odds of SSI clearly increased with age for four surgery categories (CABG, hip and knee prostheses and open reduction of fracture), but not for four other surgery categories (large and small bowel, limb amputation and vascular surgery). The results were most varied for gender. Women had lower odds of SSI for knee prosthesis and open reduction of fracture, higher odds of SSI for CABG and similar odds of SSI for small and large bowel surgery, hip prosthesis and limb amputation. Preoperative duration of stay, an additional generic risk factor identified by the reviews, was associated with an increase in the risk of SSI for the four surgery categories with the largest number of data (hip and knee prosthesis, CABG and large bowel surgery).

Conclusions

The research literature does not allow a set of surgery-specific or generic risk factors to be defined. We believe that there is a need for high-quality research to develop a revised SSI definition that has satisfactory psychometric properties and which can be applied in everyday clinical settings and to the surveillance for SSI after discharge from hospital. Research to identify risk factors for SSI needs to be carried out to higher methodological standards,primarily by following established epidemiological principles.

Surgical site infection definitions vary between surveillance programmes and,because they are complex and difficult to apply, potentially between hospitals within programmes. Definitions that are different, some in apparently only minor ways, do not have good agreement. The most widely established definitions have a similar ability to predict outcomes influenced by SSI.

In surgery-specific multivariable risk-adjusted models, associations between components of the NNIS risk index and the odds of SSI varied both quantitatively and qualitatively for different surgical procedures; this finding also applied to other risk factors investigated. There was no evidence for effect modification of risk factors by hospital.

Surveillance programmes are important to inform clinical governance and the management of infection control over time. Performance estimates (data quality and SSI per cent) based on consistent surveillance methods for institutions and groupings within institutions should be disseminated locally for this purpose. Comparisons of performance estimates (SSI per cent) for institutions or countries should be regarded with caution; nevertheless, comparisons against a benchmark may prompt institutions to make changes to infection control practices that are associated with improved performance. Judgements about the quality of medical care provided by hospitals should not be based on these statistics alone by agencies responsibility for auditing performance. National surveillance systems should comply with a set of features designed to ensure their quality.

Future research should focus on developing an SSI definition that has satisfactory psychometric properties, that can be applied in everyday clinical settings, includes PDS and is formulated to detect SSIs that are important to patients or health services.

Funding

Funding for this study was provided by the Health Technology Assessment programme of the National Institute for Health Research.

© Crown Copyright

C. Gibbons, J. Carpenter and D.L. Lamping are all based at the London School of Hygiene and Tropical Medicine, London, UK.J. Bruce is based at The University of Aberdeen, Aberdeen, UK.A.P. Wilson is based at The University College London, London, UK.J. Wilson is based at Imperial College Healthcare NHS Trust, London, UK.A. Pearson is based at The Health Protection Agency, London, UK.Z.H. Krukowski is based at Aberdeen Royal Infirmary, Aberdeen, UK.B.C. Reeves is based at The University of Bristol,Bristol, UK.