Data collection and analysis methods used in the evaluation of CQC’s new model of hospital inspection
| Method | Implementation and analysis | Participants | Topics covered |
|---|---|---|---|
| Interview | Face to face; semi-structured Conducted individually by two researchers Recorded and transcribed Thematic analysis using Dedoose | 18 key stakeholders within CQC and other national-level organisations involved in the regulation of hospitals or service improvement | The rationale for the design of the model: how it is meant to work, how it differs from previous models, the problems that it seeks to solve, any concerns, what success would look like, the expected impacts, how the impacts will be sustained over time, how the model will work alongside other regulatory processes and organisations |
| Telephone; semi-structured Conducted individually by all researchers Recorded and transcribed Thematic analysis using Dedoose | 35 inspection team members from 17 teams, spanning the range of roles and professional backgrounds | The inspection process: the usefulness of pre-inspection preparations, whether the composition of the team was right, how KLOEs were determined, the usefulness of KLOEs, how findings and ratings were arrived at, how the unannounced and announced inspections compared, how the process might be improved, interest in participating in future inspections | |
| 25 hospital staff from 13 inspected organisations – senior managers or other staff responsible liaising with CQC about the inspection, plus some operational staff in inspected service areas | The ability of the inspection to identify important performance issues and promote performance improvement: how services prepared prior to inspection, how well the inspection process worked, the accuracy of the inspection report and ratings, the impact on services and service improvement | ||
| Observation | Non-participant observation Conducted by pairs of researchers Semi-structured free text reflective summary sheet completed. Used to inform interviews and surveys; an aide-memoire for triangulation with other data | Preparation (1 day) and inspection (lasting 2-4 days) of six organisations, spanning a range of sizes, CQC risk categories and governance types. Shadowing individual inspection team members within selected sub-teams and observing on-site and off-site team meetings Subsequent single-day observations by individual researchers of inspections of three organisations to check for process changes | Inspection team expertise, use of intelligence/surveillance data, preparation and planning, logistics, inspection team (roles and responsibilities, dynamics, leadership, functioning), inspection process, use of evidence to form judgements, feedback process, provider and stakeholder engagement and response |
| Survey | Online; mix of Likert scales, tick boxes and free text boxes Personalised e-mail invitations with up to two reminders Univariate and bivariate statistical analyses using SPSS. Thematic analysis of free text data using Excel | 369 team members from inspections of 19 organisations. Response rate 66% | Motivations for joining the inspection team, the usefulness of various tools and processes designed to support the inspection, confidence in having the necessary skills to gathering information using the various mechanisms available, the accuracy of ratings, intentions to participate in future CQC inspections |
| 698 managers and senior clinicians from 18 inspected organisations. Response rate 40% | Preparations made for the inspection visit, the ability of various inspection activities to provide inspectors with accurate information, the knowledge and skills of the inspectors, how well the CQC identifies good practices and concerns, actions likely to be taken as a consequence of the inspection, the impact of those actions |
| Method | Implementation and analysis | Participants | Topics covered |
|---|---|---|---|
| Interview | Face to face; semi-structured | 18 key stakeholders within CQC and other national-level organisations involved in the regulation of hospitals or service improvement | The rationale for the design of the model: how it is meant to work, how it differs from previous models, the problems that it seeks to solve, any concerns, what success would look like, the expected impacts, how the impacts will be sustained over time, how the model will work alongside other regulatory processes and organisations |
| Telephone; semi-structured | 35 inspection team members from 17 teams, spanning the range of roles and professional backgrounds | The inspection process: the usefulness of pre-inspection preparations, whether the composition of the team was right, how KLOEs were determined, the usefulness of KLOEs, how findings and ratings were arrived at, how the unannounced and announced inspections compared, how the process might be improved, interest in participating in future inspections | |
| 25 hospital staff from 13 inspected organisations – senior managers or other staff responsible liaising with CQC about the inspection, plus some operational staff in inspected service areas | The ability of the inspection to identify important performance issues and promote performance improvement: how services prepared prior to inspection, how well the inspection process worked, the accuracy of the inspection report and ratings, the impact on services and service improvement | ||
| Observation | Non-participant observation | Preparation (1 day) and inspection (lasting 2-4 days) of six organisations, spanning a range of sizes, CQC risk categories and governance types. Shadowing individual inspection team members within selected sub-teams and observing on-site and off-site team meetings | Inspection team expertise, use of intelligence/surveillance data, preparation and planning, logistics, inspection team (roles and responsibilities, dynamics, leadership, functioning), inspection process, use of evidence to form judgements, feedback process, provider and stakeholder engagement and response |
| Survey | Online; mix of Likert scales, tick boxes and free text boxes | 369 team members from inspections of 19 organisations. Response rate 66% | Motivations for joining the inspection team, the usefulness of various tools and processes designed to support the inspection, confidence in having the necessary skills to gathering information using the various mechanisms available, the accuracy of ratings, intentions to participate in future CQC inspections |
| 698 managers and senior clinicians from 18 inspected organisations. Response rate 40% | Preparations made for the inspection visit, the ability of various inspection activities to provide inspectors with accurate information, the knowledge and skills of the inspectors, how well the CQC identifies good practices and concerns, actions likely to be taken as a consequence of the inspection, the impact of those actions |
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.