A hospital-based mixed-methods observational study to evaluate a hip and knee replacement quality improvement project Open Access

The orthopaedic service at the Golden Jubilee National Hospital (GJNH), in Glasgow, Scotland, is the recognised national centre for hip and knee replacement within Scotland. The service was set up in 2003, and the enhanced recovery pathway was initiated in 2007 following a visit by members of the inter-disciplinary team to Copenhagen in Denmark to observe a fast-track hip and knee replacement service (Husted and Holm, 2006). The GJNH team then developed a designated Enhanced Recovery Programme (ERP), which they named the CALEDonian® technique. Its implementation resulted in improvements to patient outcomes whilst reducing the length of stay following surgery (McDonald et al., 2012).

From 2010, a national programme within Scotland to establish ERP as the normal pathway of care for all patients undergoing joint replacement was launched and strongly supported by the GJNH. This programme resulted in improved patient care throughout Scotland, including reductions in urinary catheterisation use, the need for blood transfusion and the mean post-operative length of stay for patients across Scotland (Scott et al., 2013).

However, whilst the national ERP improved outcomes nationally, outcomes at the GJNH remained consistent but did not continue to improve. Therefore, with the ongoing national improvement work driving hospital boards across Scotland to improve, the outcomes at the GJNH became average when benchmarked nationally. Therefore, in order to sustain its position as a recognised national centre of excellence and to accommodate a change in referral sources and the continuing increasing demand on its services, it was vital that a review of the current enhanced recovery after surgery (ERAS) pathway was undertaken. This would help ascertain where further improvements could be made that would improve clinical outcomes and maximise capacity.

In order to instigate change, it was recognised by local leadership that a systematic quality improvement (QI) effort was required in order to understand how to make improvements relevant to the GJNH current pathway. A clinical leader from the GJNH approached an external QI researcher to help with the QI effort after participating in a workshop at the 2012 ERAS UK Conference (ERAS UK Conference, 2012). In the workshop, a QI method used to advise the implementation and improvement of an ERAS pathway in hip and knee replacement was presented. The clinical leader identified that this QI method could be used at the GJNH, and so the QI researcher was invited to visit the GJNH hospital to meet with clinical leaders and hospital executives. The QI method, which was a model to manage variability (adapted from Litvak (2005) for use in clinical microsystems to improve care processes), was presented, and the GJNH leadership team agreed to engage the QI researcher to help them with the project.

The need for external help and the need to use a specific QI method to inform change were recognised by the GJNH team due to their work nationally to help other sites implement ERAS. They recognised that process changes were required, and these needed to be driven by the whole inter-disciplinary team, and this resonated with guidance from the National Health Service (NHS) Institute for Innovation and Improvement (Institute for Innovation and Improvement, 2006). The team also acknowledged how the fast-track hip and knee replacement service they had observed in Denmark had continued to improve. Patients were now being routinely discharged at a median of two days post-surgery (Husted et al., 2011). The improvements to care achieved by the Danish team were achieved by carefully analysing the specific barriers to discharge within their clinical context (Husted et al., 2011), and areas for improvement were highlighted as improvements to care processes.

Further, it was acknowledged that if the patient pathway was highly structured and standardised, and if the inter-disciplinary team were involved in the development and production of the pathway, then improvements to patient care were possible.

Leadership at the GJNH identified that a specific QI method was required that would help to structure, analyse, implement and sustain improvements. The model to manage variability was chosen because it was identified by the project leads as a QI method that could help to reorganise care process. It was felt to have the required sensitivity and format for managing variability that could be understood by the clinical team. This was supported by the fact it had been utilised in another orthopaedic clinical microsystem successfully, and the leaders were aware of the results and process (Wainwright and Middleton, 2010).

The model to manage variability works as a QI method by identifying sources of variability within the clinical microsystem that may affect the outcome measure seeking to be improved. Clinical microsystems are an appropriate organisational level at which QI efforts can be applied (Donaldson and Mohr, 2000; Nelson et al., 2002) and defined as the “small, functional, front-line units that provide most health care to most people. They are the place where patients and providers meet, and ‘the quality and value of care produced by a large health system can be no better than the services generated by the small systems of which it is composed” (Nelson et al., 2002, p. 473). Sources of variability are identified by using a cause-and-effect diagram (or Ishikawa diagram), and then sources of variability are either classified as artificial or natural variability. This is a novel and distinguishing factor from other QI methods, such as Lean or Six Sigma. An outline of the approach is provided in Figure 1.

The model, adapted from Litvak (2005), proposes that the artificial variability of care processes is the most likely barrier to providing efficient and high quality healthcare. Natural variability is explained as being the intrinsic, normal and naturally occurring part of every system. Natural sources of variability are identified and then subdivided into “clinical, flow, and professional” categories. Natural “clinical variability” may represent the wide range of naturally occurring clinical presentations a patient may have, the level of their symptoms and their responses to treatment whereas natural “flow variability” may relate to the random arrival of patients for treatment and their consequential referral to hospital. Natural “professional variability” refers to the intrinsic differences in experience and technical skills that normally occur across healthcare professionals. If any variability is not easily classified into any of the “natural” subcategories, it is thought to be “artificial”. The rationale for identifying artificial variability is that it usually arises in processes because of the decisions made by those managing the system. It does not naturally occur, and in most healthcare systems, it is almost always multi-factorial and frequently hidden. It is therefore difficult to understand and identify without a systematic approach or method. Artificial sources of variability should therefore be removed, and natural sources of variability should be managed.

The decision to adapt Litvak's (2005) model was underpinned by a recognition that all care processes within a clinical microsystem are subject to variability and that an improvement to quality would occur through understanding and reducing the unintended variability within this system (Wheeler, 1999). Adapting Litvak's (2005) model for use to improve care processes was further thought to be attractive to clinical teams because whilst it acknowledges the need to remove unintended artificial variations in practice as a central objective, it also provides consideration of the natural differences between individual patients. This was felt to be an advantageous feature over alternative QI methods by the clinical team.

Previously, research utilising this approach has focused on modelling improved patient flow in unplanned care areas, such as critical care, emergency departments and operating theatres (McManus et al. 2003, 2004; Litvak, 2005). This adaption of the model is a novel development, and utilising the model to manage variability as a QI method to improve care processes has not been previously proposed or studied. In this case, a QI method is defined as a “systematic technique for identifying defects in clinical systems and making improvements, typically involving process and remeasurement” (Jones et al., 2016).

The objective of the project was to maximise capacity at the GJNH in order to help meet the increasing demand in Scotland for hip and knee replacement, whilst re-establishing the GJNH as the exemplar unit in Scotland for outcomes following hip and knee replacement. The project utilised the model to manage variability as a QI method, in order to inform improvements to the ERAS pathway within the GJNH elective orthopaedic clinical microsystem.

The clinical microsystem team decided on two improvement aims.

1. To ensure all patients are pre-assessed and fit for surgery two weeks prior to their operation (more specifically, to improve from the current rate of 65–100%)

2. To reduce average length of stay (LOS) for hip and knee replacement by two days (more specifically, to reduce LOS from the average of 5.5–3.5 days)

The first aim was chosen as the team recognised a high number of patients were attending for pre-assessment less than two weeks prior to surgery. This would often lead to theatre slots not being filled, cancellations when patients were found to be “not fit for surgery” and preparing patients appropriately pre-operatively with the right education and information was difficult and often rushed. Appropriate discharge planning could also be a problem, as there was not enough time to make arrangements before admission. Reducing LOS was chosen by the team as the second improvement, as they felt LOS was an appropriate proxy indicator of quality, and it was the comparator outcome measure used in Scotland to nationally benchmark.

This study and the methods used are reported in accordance with the Standards for Quality Improvement Reporting Excellence (SQUIRE) guidelines (Ogrinc et al., 2016).

The GJNH is Scotland's specialist hospital for reducing patient waiting times, and as such, as well as serving local patients, referrals are received from across the country. The hospital is a busy and large elective care centre, performing over 57,000 procedures per year, with over 1800 members of staff, more than 200 in-patient beds and 16 operating theatres. The GJNH is home to one of the largest elective (planned care) orthopaedic centres in Europe, performing over 3,500 hip and knee replacements each year. The orthopaedic clinical microsystem has an inter-disciplinary approach to care, with consultants, nurses, physiotherapists and occupational therapists all working together.

The principles of Litvak's (Boston University Health Policy Institute, 2006) variability methodology were adapted to identify, classify and manage the intrinsic sources of variability contributing to the delays in the pre-assessment process (improvement aim 1) and the current patient LOS within the clinical microsystem (improvement aim 2). By employing this framework, the team was able to co-ordinate improvement efforts.

The first stage was to undertake an analysis of current processes and to identify sources of variability that were contributing factors to delays in the pre-assessment process and the current LOS experienced by patients. This was coordinated by the two clinical managers. A workshop facilitated by the external QI researcher involving leaders from across the inter-disciplinary team was held in order to identify and agree causes of variability within the clinical microsystem related to the improvement aims. Attendees at the workshop involved representation from the pre-op, intra-op and post-op care teams and included surgeons, anaesthetists, nurses, theatre staff, therapists, radiography staff, pharmacists and administrative staff.

This workshop was held in January 2013, and the outputs were summarised by the group into a cause-and-effect diagram for each improvement aim (Figures 2 and 3). The cause-and-effect tool (or Ishikawa diagram) is considered one of the seven basic tools of quality control (Ishikawa, 1985). It is also known as a fishbone diagram because of its shape. In this case, the “fish head” represented the improvement aim. The potential causes of variability that affected the two improvement aims identified within the workshop were indicated as the “fish bones” of the diagram. Once these variables or causes of variability were identified, they were subsequently classified into either “natural” or “artificial” causes of variability according to the classification proposed by Litvak (2005).

Following the identification and classification of variability, the first active step was to eliminate the artificial variability from the care processes of the system. Litvak (Boston University Health Policy Institute, 2006) explains that artificial variability should not be accepted or worked around. Changes were therefore introduced to remove artificial variability from the delivery of care processes, and the details of what was changed, when and by whom can be seen in Table 1. Once artificial variability had been eliminated, and the causes of natural variability were managed where possible. Details of changes made to the care processes to manage natural variability are also provided in Table 1.

Once the methods to remove artificial variability and manage natural variability were identified, the new care processes were introduced. Following the initial workshop in January 2013, the change phase of the project was defined as being from January 2013 to June 2013, and this describes the period when the identified changes were prepared and introduced as they became ready. The intervention phase from July 2013 to December 2013 describes when all of the changes were implemented, and the post-intervention phase from January 2014 to December 2014 describes the period post-implementation, where there was no project activity, but outcomes were continually monitored to evaluate sustainability. This was to see whether the changes observed within the project were sustained, part of an external ongoing trend, or could be concluded to be a result of the intervention.

A before and after prospective observational cohort study design was used, with a mixed-methods sequential explanatory design (quantitative (QUAN) emphasised) that consisted of two distinct phases (Creswell et al., 2003). A quantitative phase followed by qualitative phase was used to evaluate how successful the model to manage variability was as a QI method (Creswell, 2009). In this design, the quantitative data were collected and analysed first, and then the qualitative data were collected second in the sequence, in order to help explain the quantitative results achieved in the first phase. This is summarised in Figure 4. Creswell (2009) uses capital letters to emphasise the dominant approach.

The SQUIRE guidelines (Ogrinc et al., 2016) were used as a framework to plan, structure and report the findings. Within this SQUIRE framework, the TiDier checklist (Hoffman et al., 2014) is used to describe the intervention (Table 2).

Quantitative data were collected and then analysed. It was a four-condition design: where the first condition was a baseline phase, the second condition was the change phase, the third condition was the intervention phase and then the fourth condition was a post-intervention phase. The effect of the process changes described in Table 1 were measured by collecting and analysing data for the time from pre-assessment to operation (improvement aim 1) and LOS (the outcome measure of improvement aim 2). Statistical process control (SPC) charts were used to evaluate process changes over time. At the end of this quantitative phase, the aim was to establish whether the intervention was successful or not at improving the desired outcomes within the clinical microsystem.

Following the quantitative analysis, there was a second qualitative phase of the evaluation that built on the initial quantitative phase and was connected by this intermediate stage of the study. In mixed-methods research, integration may occur via connecting, building, merging or embedding (Fetters et al., 2013). Integration in this study occurred through building, where by the quantitative data collection informed the data collection approach of the qualitative, with the latter building on the former. The rationale for this approach was that the quantitative data that were collected and analysed established to what extent the project aims were met. Once it had been established quantitatively whether the intervention was successful or not, the qualitative data and analysis in the next phase was used to explore how and why the intervention was either successful or not, and the relative role of the model to manage variability as a QI method (Rossman and Wilson, 1985; Tashakkori and Teddlie, 1998; Creswell, 2009).

This explanatory sequential design is a two phase mixed-methods design and was chosen so that the qualitative data in the second phase could help to explain or build upon the initial quantitative results (Creswell, 2009). The design was well suited to this study because the qualitative data were used to explain the outcomes of the project (Morse, 1991). It also provided an understanding from the staff perspective of how easy the model was to implement, use and manage within the clinical setting. The information generated here helped to refine conclusions regarding the model on its generalisability and subsequent potential use in other elective clinical microsystems.

In this second stage of the mixed-methods sequence, qualitative data were collected and analysed and used to help explain, and/or elaborate on, the quantitative results achieved in the first phase. This qualitative data were collected using interviews of the two clinical managers who led the implementation of the model. Interviews were conducted after completion of the intervention stage. These two individuals were tasked with leading the project and led the deployment of the model to manage variability as a QI method. They therefore had the greatest insight into its usability and success. Open-ended questions were used and informed by the mixing phase described previously. Contextual factors surrounding the implementation of the model and the project were explored. The contextual factors included in the model for understanding success in quality (MUSIQ) (Kaplan et al., 2012) were used as a prompt to ensure that all relevant factors were considered. Data collection consisted of observational notes recorded by the interviewer in addition to a recorded transcript.

The results of both the quantitative and qualitative stages were interpreted and synthesised in relation to each other and the wider evidence base. Then a summary of the most important successes and difficulties in implementing the model was made, and the main changes observed in care delivery and clinical outcomes because of the model were stated. A comparison and evaluation of the study results in light of the evidence base is made. Consideration was then given to possible sources of bias or imprecision in design, measurement and analysis that may have affected the study outcomes (internal validity). Factors affecting external validity such as the generalisability of the model were also considered. Consideration was also given in relation to the sustainability of any changes, i.e. the likelihood that any observed gains might weaken over time.

The outcome measures used to evaluate the outcome of the intervention in relation to each improvement aim are provided below.

1. In relation to the first improvement aim, time from pre-assessment appointment to operation date was calculated (days were measured as the number of midnights between the pre-assessment appointment and day of admission to hospital).

2. In relation to the second improvement aim, LOS in hospital was calculated (days were measured as the number of midnights between day of admission and discharge from hospital).

For both outcome measures, data were extracted from the hospital administrative data system and checked for accuracy and completeness against local (clinical microsystem level) audit data. LOS is commonly used as a proxy indicator of quality and is the widely used outcome measure for the implementation of ERAS. In regard to this project, both outcome measures were considered relevant, reliable and valid outcome measures that were feasible to obtain and had good usability.

For the quantitative data, SPC was used to monitor the change in outcome measures for both improvement aims. Change was evaluated between each of the four project phases (baseline, change, intervention and post-intervention stage. For both outcome measures, the data were considered to be continuous data, and so the xmr chart was judged the appropriate SPC to use (Mohammed et al., 2008). Both outcome measures were evaluated by calculating the mean on a monthly basis and then presenting as monthly data in consecutive points. Data for the six months prior to the project (July 2012–December 2012) is presented as the baseline phase, along with data from the start of the project for the next two years (January 2013–December 2014), to cover the change, intervention and post-intervention phases as described previously. The data from the baseline and project phases are presented continuously, and the mean and control limits were re-calculated at the start of each phase.

For the qualitative data, the process of analysis started with familiarisation of the data, before organising and preparing the data for analysis. Thematic analysis was then undertaken through a process of coding themes from the interviews relating to the contextual factors included in the MUSIQ (Kaplan et al., 2012). The data were then interpreted in light of both the quantitative and other qualitative findings.

Thematic analysis was chosen as a method because of its flexible approach that could be modified to the need of the study, whilst providing a rich and detailed account of the data (Nowell et al., 2017). The aim of the analysis was to enable an understanding of how the intervention worked or failed to work from the perspective of the individuals involved in leading the project. There were six phases to the analysis, as recommended by Nowell et al. (2017). After familiarisation with the data, initial codes were then created (accompanied by reflexive journaling), and then themes were searched for. These themes were then reviewed and triangulated, before they were defined and named.

The project was presented to the Head of the Research Department at the GJNH in November 2012. It was defined as a QI activity; therefore, the need for submission to the hospital and local NHS Research Ethics Committee was confirmed as not being required. However, full ethical consideration was given to the project by utilising published guidance and policy templates from the Healthcare Quality Improvement Partnership (HQIP). This ensured that the patients' interests and rights were properly protected throughout the study. The HQIP template provided outlines for best practice structures and mechanisms that provided an ethical oversight and formed the basis of a thorough governance framework.

Following the decision to improve the service, the external QI researcher was invited to facilitate an introductory workshop in January 2013. This was to meet staff, initiate the project and introduce the model to manage variability. It was also important for the external QI researcher to establish credibility with the local team and to start to build relationships with staff. Internally, the project was supported by an executive sponsor, two clinical managers with service improvement experience and lead clinicians from surgery, anaesthetics, nursing and therapies. The two clinical managers led the QI effort locally and coordinated the project team. This core team was supported externally by the QI researcher who over the course of the project made 12 site visits to the hospital (every 2–4 weeks) and also assisted remotely.

The initial workshop in January 2013 was followed by a change phase of the project, defined as being from January 2013 to June 2013, and this describes the period when the identified changes from the workshop were prepared and gradually introduced as soon as they ready. The intervention phase from July 2013 to December 2013 describes the period when all of the changes were implemented and regularly monitored by the project team, and the post-intervention phase from January 2014 to December 2014 describes the period post- implementation, where outcomes where monitored but the formal project had finished.

The two clinical managers leading the project both worked part-time on the project around their normal duties and led the local team through the use of the model to manage variability. Staff members from all professional groups attended the workshops and contributed to a cause-and-effect diagram, which aimed to identify all of the causes of variability that contributed to the short period of time between pre-assessment and admission, and the current LOS which was felt could be reduced. The core team then pulled out the key themes and factors that they felt most strongly influenced the current outcome measures of the two project aims following this consultation process. Where data were required to analyse specific factors, they were extracted from the hospital patient record system.

Following the identification of factors that were influencing both improvement aim outcome measures, the team decided on the changes required to care processes in order to either remove artificial variability or manage natural variability. The results of this process are summarised in Table 1. The team then planned how the necessary changes would be implemented and agreed to initiate all of the changes as soon as they were able, within the change phase (January 2013–June 2013). It was decided that the new care processes that made up the intervention would all be in place by July 2013 and would be actively monitored by the project team from July 2013–December 2013. Once the change in care processes was introduced, regular review using SPC was undertaken by the project team to monitor performance.

SPC charts are presented (Figures 5–8) for the outcome measures defined for both improvement aims. They show data for the baseline phase, change phase, intervention phase and post-intervention phases. The data are continuous and account for all patients operated on within that time period. There is no missing data. In order to present the data cleanly, and in accordance with routine outcome monitoring at the hospital, monthly means were calculated in order to produce the xmr chart. The xmr chart consists of two charts, the x-chart and the mr-chart. The x-chart is a control chart of the 30 observed values for each outcome, and the mr-chart is a control chart of the moving ranges of the data. At each phase, the control chart is recalculated, and the processes remain stable within each phase for both outcome measures.

Following the quantitative analysis, the results were shared with the two clinical managers, and the second qualitative phase of the evaluation was conducted. Integration with the mixed methods design occurred through building, where by the quantitative data collection informed the data collection approach of the qualitative interviews. The qualitative data collection and analysis aimed to explore how and why the intervention was either successful or not and the experience of utilising the model to manage variability.

Both interviewees felt the project had been successful, in which the procedural elements of the QI method (the model to manage variability) had been executed as planned and that the outcomes had improved. However, there was acknowledgement that the team had not managed to change all of the care processes identified by the model and had not achieved their initial improvement aims entirely. Both interviewees judged success based on the relative improvement to the outcomes measures linked to the improvement aims. This was opposed to reviewing whether each change to care process had been achieved, or whether there had been any other wider benefits of the project.

Consequently, the viewpoints of the two clinical managers leading the project were explored to understand why the project had not been more successful.

The role of utilising the model to manage variability as the QI method used to inform the intervention was explored with the interviewees. Both interviewees viewed the model positively and did not think that the model was associated with why the improvement aims had not been fully achieved. Specifically, they both reported advantages of the model and found it intuitive to use.

Advantages of the model over alternative QI methods that both interviewees had previously used were also highlighted.

And both interviewees described how the model had impacted their other work in the hospital, either by reporting that they had used the model in other projects or by describing how the concepts within the model where now used routinely by other members of staff.

Both clinical managers felt that the model to manage variability was easy to use, had been implemented appropriately and had impacted the organisation more widely that just within this specific project. Therefore, the relative roles of other contributing factors to the projects outcomes were discussed. Contextual factors thought to both facilitate and limit the project were considered.

Both interviewees thought that the role of the external QI researcher and the credentials and expertise of that person played an important facilitative role within the project.

Conversely, when invited to discuss why the project had not been as successful as planned, both interviewees perceived that increased leadership and engagement from within the clinical microsystem would have improved outcomes. They reflected on the role of leadership from a personal perspective and also in relation to the participation and leadership of others involved in the project.

In addition, there were other organisational factors that were highlighted by both interviewees in regard to staffing. They both noted frustration that it had been hard to adequately organise anaesthetic cover within the pre-assessment clinic in order to increase the number of day of surgery admissions. They explained that the human resource process to change to a seven-day therapy working model had also taken longer than anticipated. One of the clinical managers noted a frustration in regard to knowing what they wanted to implement but not being able to do it.

A successful QI effort is one in which the intended improvements are successfully achieved. In this project, the primary outcome measures associated with each improvement aim improved, although they did not meet their targeted improvement. For improvement aim one, the percentage of patients who had more than two weeks between their pre-assessment clinic visit and their operation increased from 65% in the baseline phase to 76% in the intervention stage. This 11% improvement then reduced by 3%–73% in the post-intervention monitoring period. All of the changes to the care processes identified as being required to increase the percentage of patients who had more than two weeks between pre-assessment clinic visit and their operation were made. However, the 100% aim was still not achieved. This outcome may have been due to the fact there were actually not enough patients on the waiting list at any one time, in order to plan operating lists more than two weeks in advance.

For improvement aim two, mean LOS decreased from 5.4 days in the baseline phase to 4.7 days in the intervention stage and 4.5 days in the post-intervention monitoring period, representing an 18% overall decrease in LOS. This reduction, is analogous with reductions reported in other implementation studies of ERAS in joint replacement. Saunders et al. (2016) reported a 17% reduction in LOS for primary joint replacement following the introduction of an ERAS pathway, and a reduction of 19% was reported by Ricciardi et al. (2020) in their report of utilising lean as a QI method when seeking to improve a knee replacement pathway. It should also be noted that in a recently published report from NHS England (GIRFT, 2020), the national average length of stay reduced for hip replacements by 19% and for knee replacements by 17.8% in the period 2014–2019. The improvement made by the team at the GJNH within the project could therefore be argued to be highly significant compared against the background trend LOS. It was also lower than the national average in 2014 for both hip (5.18 days) and knee (5.25 days) replacement in Scotland (Scottish Arthroplasty Project). There was also and importantly no statistical or clinically significant change to balancing measures, such as major complications after surgery (including dislocation, infection of the operated joint, deep vein thrombosis or pulmonary embolism (DVT/PE), death, acute myocardial infarction (AMI), acute renal failure and cerebrovascular accident (CVA) or stroke) throughout the periods of data collection. These remained within expected values as measured and reported by the Scottish Arthroplasty Project (2019).

The reasons for failing to achieve a greater reduction in LOS may be assumed to be because the proposed changes to clinical processes (informed by using the model to manage variability as a QI method) that made up the intervention were not all implemented as planned. There were difficulties and delays to organising and changing the artificial variability of staffing levels. The seven-day therapy service took longer than anticipated to implement, and the provision of anaesthetic cover within the pre-assessment clinic was difficult due to inadequate staff numbers and difficulties with scheduling rotas. These were both changes to the care processes that aimed to decrease LOS by increasing the rate of day of surgery admission (as anaesthetic review would be undertaken in clinic rather than on admission) and to expedite early mobilisation and avoid delays to discharge (through earlier access to therapy). This difficulty to implement post-operative elements of an ERAS pathway is not uncommon, and it has been found in reviews of ERAS implementation that postoperative elements related to mobilization and rehabilitation often demonstrate much lower levels of compliance compared to other stages of the peri-operative pathway (Coxon et al., 2017).

The mixed-methods approach to the evaluation, accompanied by the explicit reporting of the intervention through the utilisation of the TIDieR checklist, is a strength of the project. QI reporting in surgery is acknowledged to be generally poor (Jones et al., 2016) and the explicit reporting of intervention and QI method used, along with context has been recommended (Jones et al., 2019). Research evaluating QI success is strengthened by utilising approaches from the social sciences, such as mixed-methods (Kaplan et al., 2010). In this case, the connecting component and qualitative interviews conducted with clinical managers provided experiential data from those involved in leading and facilitating the project to explain some of the specific contextual factors that influenced the outcomes of the project.

This qualitative data confirmed that the two clinical managers felt the project had been successful, although only partially. They thought that it had been leadership, staffing and organisational related issues that had prevented the outcomes improving further. Such issues are consistent with the wider QI literature (Kaplan et al., 2010) and experience of teams working to implement ERAS pathways (Paton et al., 2014). It may be argued that more qualitative data could have been collected from the wider team (including clinical staff), and this could have confirmed that saturation was achieved. However, due to the number of people involved across the peri-operative pathway, it was felt that if more value was to be gained, then at least one person from every department/profession would need to be interviewed, and time and resources did not allow for the additional 10–20 interviews. Additionally, as this research also sought to evaluate the use and deployment of a novel QI method (the model to manage variability) to guide the improvement effort, it was felt the feedback and views from the two clinical managers facilitating the use of the QI method would be most pertinent.

In regard to the QI method used, the two clinical managers highlighted that the model to manage variability had been received well by the inter-professional team. It had engaged them in the QI process and led to the technique being used in other projects within the hospital, and the terms natural and artificial variability had made their way into common usage amongst the team. More specifically in relation to the model to manage variability, both clinical managers thought it offered advantages over other QI methods they had used in the past, and both felt that the external input of the QI researcher to help introduce it was an important facilitating factor.

These findings confirm the evidence from the wider evidence base where it is acknowledged that the use of a QI method (in this case the model to manage variability) can be helpful to inform improvement efforts. However, it was other contextual factors that were highlighted by the two clinical managers as being key contributing factors to the project outcome. Context is well understood to be a critical factor in QI research (Stevens and Shojania, 2011), and despite the GJNH being a recognised national centre for hip and knee replacement, it was hard for the team to lead and manage change within the organisation, in order to make the required staffing changes and drive the project forward.

Knowing how to improve is not always the same as being able to improve. Successful implementation of ERAS pathways is known to be associated with an organisation having a change agent to fully drive the implementation process (Roberts et al., 2010; Coxon et al., 2017). Given that both clinical managers acknowledged that they could only dedicate part of their time to the project, it may be judged that the absence of a full time change agent with overall responsibility for the project was a contributing factor to the only partial achievement of the project aims.

In closing, this novel QI method, similar to alternative QI methods (e.g. Lean and Six Sigma), has a strong theoretical theory to underpin its use. Further, this study provides supportive empirical evidence to illustrate that the model to manage variability may be an effective QI method to guide improvement efforts within an NHS clinical microsystem and may offer advantages over other QI methods. However, the study also provides an important insight on the connection between applying improvement science theory to a specific case scenario (or clinical microsystem) within the real world. The explanatory data from the interviews highlight the complex relationship between context, mechanisms and outcomes when conducting QI work. This is an important finding and confirms the future need for a greater use of behaviour change and organisational psychology theory to improve the design, adaptation and evaluation of QI methods in healthcare.

In regard to the generalisability of the work, as with many QI reports, the failure to include a comparison group means that external causes for change cannot be ruled out. However, the attempts made to ensure transparency within the reporting and the mixed-methods explanatory design should be highlighted as efforts made on behalf of the reader to counteract this potential bias. This is important because consideration should always be given to possible sources of bias within the design and reporting of a study. It is therefore acknowledged that the role of the QI researcher as both an external change agent within the project and as the researcher evaluating the project is important to recognise. The need to acknowledge reflexivity is an accepted issue within the reporting of many QI studies, and so thorough reporting of the details of implementation and evaluation is very important. In the absence of external and independent evaluation (which of course may also introduce its own bias), transparent and thorough reporting allows the reader to make their own judgements.

This study sought to improve clinical processes within an orthopaedic clinical microsystem and to ascertain details of when, how and why the model to manage variability should be used as a QI method. The mixed-methods approach revealed in the first quantitative phase that the outcome measures for each project aim were improved, although only partially. The secondary qualitative phase which built on the initial quantitative phase provided insight on the generalisability of utilising the model, by helping to understand its implementation and other contextual factors. The model to manage variability was felt to be utilised successfully to inform the planned interventions; however, contextual factors relating to leadership, staffing levels and organisational factors meant that not all of the interventions were implemented. This provides further information in regard to the model to manage variability, in which it can be considered a useful QI method. However, as with other QI methods, it is not independent of contextual factors, which can influence the relative success or failure of the planned interventions following its use.

The authors would like to thank the staff from the GJNH who were involved in this project, and especially Christine Divers for all of her hard work and leadership.