Chapter 14: Usability of Enhanced Decision Support and Predictive Modelling in Prostate Cancer

Prostate cancer accounts for some 7.8% of all cancers worldwide and 15.1% of all male cancers [10]. It ranked as the most common cancer among men in the European Union (EU) in 2012 and among the top four most costly cancers in the EU. Furthermore, its incidence is predicted to rise, with major differences between countries [1]. This represents a significant burden on healthcare services. Beyond the generic challenges of cancer treatment – namely, early diagnosis increasing successful outcomes and attempting to predict the longer-term prognosis for the patient – prostate cancer poses specific problems for both patient and clinician. Since the case fatality rate for prostate cancer is low and disease progression is slow, there is a low tolerance for side effects of treatment. Furthermore, tumour location is problematic, and the cancer implicates multiple disciplines beyond pathology, including urology, radiology, and of course oncology, among others. Its diagnosis and associated treatment plan therefore require collaboration across departments. Successful care is dependent on multiple factors, though feedback is sparse and unstructured, with patient-generated data particularly underdeveloped. The latter is particularly relevant. Treatment is ultimately about patient satisfaction and postoperative quality of life. But these are dependent on oncological outcomes (residual cancer) as well as functional outcomes (potential incontinence and sexual dysfunction).

For some time now and before the advent of big data technologies, researchers have been aware that the introduction of new technologies as part of a clinical intervention requires careful management, specific to the complexity of a clinical or healthcare setting; it is not simply a question of robust and effective technology, for example [2–4]. To begin with and beyond the innovative technology, communication and decision-making strategies, as well as the social context for the innovation, need to be considered [4]. Subsequently, the focus has been on encouraging the engagement of appropriate stakeholders (known as ‘cognitive participation’), [3] or the stages and stakeholder discussions required to not only accept the new technology-based intervention but also ensure its long-term adoption [2]. Although not specifically intended as a critical evaluation of these different perspectives, the work reported in this chapter offers an opportunity to explore some of the diverse factors involved in introducing a technology-enhanced innovation into standard clinical practise. We concentrate here on the combination of predictive modelling in decision-making for prostate cancer, whilst also gaining some insight into the main stakeholder attitudes (patients and clinicians) towards an enhanced Clinical Decision Support (CDS) system, within the context of prostate cancer care in a nationally recognised center for tertiary care.

Led by the Karolinska University Hospital (KAR) in Stockholm, Sweden, this prostate cancer exploratory study, therefore, set out to enhance patient outcomes and increase productivity in the health sector through the application of big data technologies for predictive modelling and the integration of advanced visualisation techniques applied to complex datasets to support and encourage cross-disciplinary consultation. At the same time, of course, health data are almost exclusively specialcategory personal data irrespective of their provenance, including several medical and non-medical disciplines: urology, oncology, pathology, radiology, nursing, health economics, and patients themselves. In consequence, the security of these data and guaranteed privacy for patients are of paramount importance not only to ensure regulatory compliance but also to enhance patient trust.

The research study reported in this chapter was based on the design of an updated CDS system based on the IntelliSpace Precision Medicine (ISPM) Prostate from Philips,ⁱ implemented at KAR. Since the appropriate treatment plan is typically the result of a MultiDisciplinary Therapy conference (MDT), the CDS was intended to support primary treatment decisions in a pre-prostatectomy MDT setting at KAR. A schematic diagram is shown in Figure 14.1, where the decision dashboards represent the clinicians’ perspective. In addition, any captured big data were used to create decision models to improve oncological and functional outcome predictions after primary intervention. Results associated specifically with the introduction of the updated CDS have been reported elsewhere [5]. It is worth noting, finally, that during the lifetime of the project, the COVID-19 pandemic introduced significant and unforeseen consequences for healthcare services and the social context within which those services are delivered. Although the strategy in Sweden involved social responsibility, including social distancing, there was no imposed lockdown as in other countries. This is important because it affected working practices for clinicians dealing with coronavirus patients without the socially isolating context for other patients.

In order to evaluate the potential for the enhanced CDS and for big data prediction modelling, a comparative study was carried out at KAR comparing two cohorts and their data:

• Baseline: a group of 924 patients at the hospital between Q1 2017 and Q3 2019

• ISPM: a group of 498 patientsⁱⁱ at the hospital between Q4 2019 and Q1 2021

These represent opportunity samples from the hospital, were not at this time screened for particular characteristics, and consisted entirely of patients planned for prostatectomy. The baseline group demonstrates current practice at the hospital, whilst the ISPM group demonstrates the proposed approach with the introduction of the new technology. The work reported in this chapter represents some of the preparatory work for the final version reported in [5], and it highlights some of the challenges associated with this kind of evaluation.

For each patient, the enhanced CDS needed to be populated with data from the patient records. Therefore, data had to be imported in the first instance as part of the ISPM system implemented at KAR. Each week, patient data for around 5–15 patients were collected prospectively to populate the ISPM Prostate CDS system. A proof-of-concept for this integration was demonstrated by importing basic patient data (e.g., patient name, Swedish personal identification number – uniquely identifying the patient, date of birth, patient age, and Prostate-Specific Antigen (PSA) data) to ISPM Prostate from KarDa (the KAR datalake). At the same time, well-structured data, for instance, including surgery planning systems, financial records, and the Electronic Medical Records (EMR), were exported to KarDa. Semi-structured data, such as pathology biopsy reports, were also accessible via the same route. Figure 14.2 shows a schematic of the data architecture, including the various data import and export pipelines that have been implemented.

KarDa is the Microsoft SQL server at KAR. The following tables were imported to KarDa:

• Patient contains basic demographic information such as age, name, etc.;

• PSA contains one or more PSA test results;

• BiopsyReport contains any pathology reports for the patient as the result of any biopsies. As such, it contains semi-structured data on tissue samples and therefore the aggressiveness of the cancer;

• HealthDeclaration contains patient responses to the initial health screening questionnaire;

• SurgeryReport, a detailed report on any surgeries, scheduling, those attending, etc., from the surgery planning system Orbit;

• PathologyReport, a detailed report from any prostatectomy carried out at KAR, including detail on the primary tumour and any metastases.

From left to right in the figure, three data sources are shown: TakeCare contains the patients’ EMR, Sympathy the pathology reports, and Orbit the surgery schedule. These data sources are queried by scheduled tasks for extraction and import into the KarDa. The ISPM Prostate system collects data from KarDa into a prostate data model and stores the result in the Fast Healthcare Interoperability Resources (FHIR) database.

The ISPM Prostate system scheduler has the capability to automatically send data to the INCAnet server using a RESTful API. INCAnet serves as a proxy to validate the patient data before sending it on to the National Prostate Cancer Register (NPCR) in Sweden. OncoPredict is an R-based Shiny application supporting population analytics directly on the FHIR database via a token-secured RESTful API. OncoPredict was therefore used to monitor real-time completeness and validation of the data, deploy risk models, monitor discriminating performance of risk models, and calculate real-time risk for the patients who would be discussed at the MDT meeting.

In total, 689 patients were collected in the ISPM software for patients that were scheduled for prostatectomy. For those patients, clinical diagnostic information such as PSA, DRE, MRI data (e.g., PI-RADS scoring and ADC values), and biopsy information were collected. Similarly, post-surgical outcomes for the subset of patients who had undergone surgical treatment were collected to enable predictive analytics. The completeness of relevant information is shown in Figures 14.3 and 14.4. 

For our study, data on patients being prepared for radical prostatectomy were organised as follows:

• Procedure: the patient had undergone prostate biopsy, in a systematic and/or targeted way.

• Descriptors: completeness of data was analysed for PSA, clinical stage assessed during DRE, PI-RADS v2 score from MRI (PI-RADS), median ADC value measured in the index lesion, systematic biopsy performed (Systematic Bx), Gleason Grade Group from systematic biopsy obtained tissue (sys Bx GGG), image guided biopsy performed (image guided Bx), and Gleason Grade Group from image guided biopsy obtained tissue (Img Bx GGG) (see Figure 14.3).

• Outcome: pathologic Gleason Grade Group (pGGG), pathologic T stage (pT), Pelvic Lymph Node Dissection (PLND), pathologic lymph Node involvement (pN), and Surgical resection Margin (SM); see Figure 14.4 for completeness of data. Note that for all patients, there is a record if they had a PLND, but for those without PLND, no follow-up date of Lymph Node Involvement (LNI) is known.

As with other studies in this volume, data cleaning, imputation, and the synchronisation of data from different sources were required on occasion. The implemented solution relies on manual entry of radiology data in ISPM, whereby structured radiology reports are automatically rendered for further use, e.g., automatic writing to the EMR or transfer to registries. In the present research setting, the radiology data generated in ISPM are being used for visualisation at pre-treatment conferences and upon treatment execution as well as for predictive analytics. Automating manual data entry has continued beyond the end of the project to improve the CDS system’s usability.

To date, there have been many predictive models proposed within prostate cancer research, though very few are in clinical use. The lack of clinical adoption is attributable to three main reasons:

1. Generalisability: Models generated for one population often perform poorly on another, and so they need to be validated and adapted to the local patient population.

2. Data Heterogeneity: Diagnostic and outcome data are exceedingly heterogeneous and rarely conform to a (structured) format suited to predictive modelling.

3. Process Integration: ICT systems delivering the data and predictive modelling to the point of treatment decision and execution are lacking, sometimes hampered by local governance and control procedures.

Thus, an automated system to capture, compile, and visualise structured diagnostic and outcome data combined with predictive modelling capabilities constitutes a prerequisite for daily use in a clinical setting. In our prostate cancer study, a selection of pre-existing predictive models was chosen as proof-of-concept. With that in mind, the framework used in our study can be employed to adapt any existing models or generate entirely new models using a variety of techniques from classic mathematical modelling up to unsupervised AI methods. For instance, the prediction of prostate cancer LNI as devised by Briganti et al. was adapted to the local patient population at KAR as described below in the results Section [6].

Beyond specific big data issues, there are always challenges when assessing technologies and potential process changes in a clinical setting. Patient outcomes remain the most significant focus: there must be tangible benefit to the patient as identified not only through clinical outcomes but also regarding patient perceptions. One of the challenges here, of course, was that the ISPM timeframe includes the onset of the COVID-19 pandemic. This affected healthcare services in general, but also patient confidence. At the very least, this may be a confounding factor in any patient-reported survey. Second, taking consecutive time periods may coincidentally include changes and improvements to clinical practice independent of any planned benefit of technology.

Other factors that must be addressed include the financial cost or savings associated with the introduction of the change. In Sweden, for example, healthcare is primarily funded through taxation. Justifying costs is therefore a significant issue. But in addition, and as discussed in [2–4] and in Chapter 26 (Technology acceptance in healthcare), any change to a clinical care process may be disruptive, affecting not only patients and clinicians in the first instance but also other stakeholders.

To address these issues, a combination of quantitative and qualitative methods was used to provide some indication of the success of the exploratory work as implemented in the clinical setting at KAR.

In this section, we present some preliminary results in relation to the big data approaches to the data imported into the enhanced CDS system, as represented in Figure 14.2. This is followed by a summary of the quantitative and qualitative surveys carried out during the evaluation period.

Here, we detail some of the data selection used for the study and some of the analyses carried out with these data. First, we consider the recommendation for extending surgery to include a PLND, following official guidance. We then show some of the predictive analyses associated with metastasis prediction. Both illustrate the types of big data analyses used to pre-process the data feeding the enhanced CDS.

The first example of pre-processing involved the prediction of additional surgery to be performed for some patients; in this case, it appears that other structures outside the prostrate are involved. The European Association of Urologists (EAU) recommends prostatectomy surgery be extended with a PLND when the Briganti nomogram predicts a positive LNI above 5%. Out of the total selected data, 248 patients had a PLND, information on LNI was available, and the input variables to calculate the Briganti risk score were complete. The KAR strategy therefore involved a selection based on MRI information, the Briganti LNI risk score, and patient preference. Some men actually prefer to undergo a PLND, although the risk of LNI is low, while others decline PLND despite the high risk because of the potential side effects of the procedure. The results are summarised in Figure 14.5. This resulted in 26 patients who indeed had True Positive lymph nodes (TP) and in 222 men to whom False Positive lymph nodes (FP) were found. The EAU guidelines recommend use of the Briganti nomogram at a cut-off value of 5% (selection criteria Briganti at 5% cut-off) and results in 3 False Negatives (FN) and 95 True Negatives (TN), meaning that 95 men could have been spared a PLND at the cost of missing three patients with positive lymph nodes.

Ultimately, the MDT will decide if the PLND will be recommended. However, providing this analysis to the team should increase the richness of the data they have available to make those decisions in a timely manner.

As mentioned previously, the OncoPredict module includes risk prediction. With this in mind, the 2012 Briganti nomogram needed to be adapted for the current dataset [6]. First, an external validation of the 2012 Briganti nomogram was performed using the area under the Receiver Operating Curve (ROC) (labelled AUC in the figure) as a performance indicator. The external validation included 248 complete patient cases where the input parameters PSA, clinical T-stage, biopsy primary and secondary Gleason scores, number of biopsies, number of positive biopsies, and the MRI PI-RADS scoring were known. Figure 14.6 shows that the Briganti nomogram was able to predict LNI with an AUC of 0.73. A calibration plot showing the predicted probability of LNI in patients compared to the actual percentage of patients with LNI is shown in Figure 14.7.,Figure 14.8 shows the decision curve that demonstrates the net benefit associated with the use of the Briganti 2012 nomogram.

Second, the Briganti probabilities were calibrated by training a logistic regression model to predict the true class of a sample as a function of the uncalibrated class probability. Figure 14.6. shows that calibration does not improve the discriminating performance, most likely because at higher probabilities the model is poorly calibrated.

Next, a Random forest classifier was trained with the same input variables as the Briganti nomogram, and the discriminating performance was estimated by means of a leave-one-patient-out cross-validation [8]. Figure 14.6 shows that the Random forest classifier can predict LNI with an AUC of 0.79.

Finally, the same Random forest classifier was trained but now included the PI-RADS scoring as an additional input variable. The discriminating performance was estimated by means of a leave-one-patient-out cross-validation. Figure 14.6 shows that the Random forest classifier can predict LNI with an AUC of 0.78, indicating that PI-RADS scoring did not improve the discriminating performance.

A Random forest classifier was trained using a preselected clinically relevant set of 24 variables to predict

1. LNI and

2. ExtraProstatic Extension (EPE).

Although the Random forest classifier has an internal ranking of feature importance, low-ranking features can still have a negative effect on the overall discriminating performance of the model. Therefore, an automatic iterative exclusion of low-ranking features was used to collect the optimal set of features using the area under the receiver operating characteristic curve as an optimisation strategy.ⁱⁱⁱ

The outcome classes of LNI and EPE are imbalanced as a limited amount of prostate cancer patients have EPE and only a small proportion have LNI. Therefore, the Random forest was trained by balanced bootstrapped undersampling of the minority class, see also [5]. The difference is that the complete Random forest is trained on a bootstrap sample, and the data are bootstrapped until all patients from the majority class have been selected. The resulting set of Random forests is then combined into a single one. The prospective performance of the Random forests was estimated by means of leave-one-patient-out cross-validation. A bootstrap resampling approach with 10,000 iterations was used for estimating the bootstrap mean AUCs and 95% confidence intervals.

We applied Natural Language Processing (NLP) technology to cater for the various freeform reports included in patient records. NLP algorithms were specifically developed for this purpose and tested against two test sets of pathology biopsy reports from patients previously treated at KAR. The success rate of these algorithms was limited and very much dependent on both the content and language quality of the reports.iv Some variables (like Gleason score) were detected with a high recall rate (i.e., high sensitivity) and with high precision (i.e., high positive predictive value), typically above 80%–90%. Others (like tumour stage) were frequently overlooked (i.e., lower sensitivity) or interpreted incorrectly (i.e., lower positive predictive value). The main reason for these outcomes was the significant variability in reporting and almost complete lack of structure in a large proportion of the reports. KAR is primarily a treating hospital, not a diagnosing hospital. In consequence, most biopsy reports are created in outpatient clinics, and therefore, KAR has very limited opportunity to dictate the rules for pathology reporting.

Implementing these techniques and validating them in this way allowed for appropriate analysis of the data available prior to ISPM implementation. In the next paragraphs, we consider the response to including the big data-driven enhanced CDS into the MDT for prostate cancer patients at KAR.

Table 14.1 summarises the observed outcome measures for prostate cancer patients at KAR. Although not statistically significant, there is a reduction in incidence of these complications in outcomes after prostatectomy (as shown in the final column in the table labelled ‘Delta’).

In addition to the objective measures summarised in Table 14.1, patient perceptions were investigated. Table 14.2 summarises patient perceptions on two particularly important dimensions: the ability to hold back urine and to achieve and maintain an erection, specifically how much the patient is bothered by sequelae related to these functions. Again, the final column is illustrative only. Recognising that such results reflect only one perspective on the technology-enhanced intervention, it is important to consider the subjective patient perceptions of the clinical outcomes as they affect them.

The other major stakeholder group, beyond the patients themselves, are the clinicians. By definition, the MDT comprises experts from different disciplines. Facilitating discussion between them in terms of the information they receive as well as the process of moderating and engaging in MDT discussions are both relevant. The efficiency of the MDT was measured as Time spent at the MDT (seconds). The quality of the MDT was evaluated by external viewers using a modified version of the validated MDT-MODe metric on a 3-step Likert scale as follows:

1. No knowledge was available (1p )

2. Vague first-hand or strong second-hand knowledge was available (3p )

3. Comprehensive first-hand knowledge was available (5p )

Table 14.3 summarises the findings.

For Patient’s view, a score of 5, for example, would mean that there was comprehensive first-hand knowledge of what the patient wanted and felt was available in the MDT. Conversely, a score of 1 would mean no such information was available. With these scores, a Mann–Whitney U statistic was calculated (not shown) as an indicator of the significance of the difference between the baseline and ISPM cohorts.

Metric labels that have been italicised do not show a significant change. For all other metrics, there was a significant improvement with the introduction of the enhanced CDS. This is an encouraging result, though further investigation is required, firstly to establish the significance of individual factors and secondly to determine their effect on MDT members and their decision-making.

The reduction in time spent per patient can be assigned a rough monetary value with the assumptions that ∼10 clinicians discuss ∼10 patients in each MDT, and there are 60 similar weekly conferences in the Theme Cancer at KAR.

As a consequence of the COVID-19 pandemic, the availability of staff to provide feedback on the enhanced CDS was restricted. Nevertheless, a small representative cohort of nine urology consultants and one radiology consultant responded to a brief survey designed to capture their experiences of the CDS as part of the MDT. In Table 14.4, responses had been recorded originally on a 5-point Likert scale, labelled as shown in the table. For simplicity, results were reduced to three values summarised as positive (+), neutral (0), and negative (−). Positive summed responses for “Strongly agree” together with “Agree”, “Much higher confidence” and “Higher confidence”, or “More satisfied” and “Satisfied” respectively; negative correspondingly “Disagree” with “Strongly disagree”, “Less confident” and “Much less confident”, or “Dissatisfied” and “Very dissatisfied”.

Overall, and although the responses came from a small cohort, responses to the qualitative assessment statements regarding the use of the enhanced CDS as part of the MDT were positive: 63 responses out of a total of 89 in Table 14.4. Note, however, that “I could easily perceive the treatment recommendations made within the MDT” did not elicit an unequivocally positive response, suggesting that there may still be work to be done to improve the comprehensibility of outputs.

In addition to the general agreement/satisfaction questions in Table 14.1, when asked: “Compared to the traditional MDT format, the discussion length per patient when using ISPM is…” eight respondents thought discussion length to be shorter, and two that it was a similar length; no one claimed it took longer. Finally, in response to the question: “In order to build a comprehensive overview of the case in your mind for staging and treatment decisions, which of the following would you prefer as a way of working”, nine said that they preferred working with the enhanced CDS, one with an unspecified other method, and no one responded that they preferred the traditional way of working.

These results are encouraging. However, there is some scope to investigate further how the response wording has been interpreted. Specifically, comparing Tables 14.1 and 14.4, it will be important to reflect how the same stakeholders respond to related issues.

In this chapter, we describe the baseline (before use of ISPM) and follow-up measurements (after implementation of ISPM) of pre-prostatectomy MDTs, the prostate-cancer patients discussed at these conferences, and the staff satisfaction in relation to data presentation at the MDTs. While the introduction of ISPM was associated with higher MDT conference quality and efficiency, no statistically significant change in patient outcomes could be seen. Overall, we have observed positive staff feedback on the use of the technology during the MDT.

From our experience in this exploratory study, we would highlight the following:

1. Introduction of CDS technology may save time in the MDT setting, but to achieve an overall efficiency gain in a clinical setting, system integration is an absolute must.

2. NLP may have some benefit. However, there needs to be careful consideration of the format (and variability) of the source data, and the intended use of the NLP-generated structured data set.

3. We recommend that specific resource be devoted to high response frequency for patient-reported data.

The rationale for each of these recommendations is outlined below.

System integration: To achieve efficiency gains from CDS technology, data must be automatically retrieved from source systems, enriched using the CDS at the point of care, and subsequently made available for downstream clinical applications and secondary uses such as research, quality assurance, predictive modelling, and feedback learning. Significant effort should be made to avoid manual transfer of data, which is known to introduce errors, delays, loss of data resolution, reduced staff satisfaction, increased staff turnover, and increased costs. It can also be argued that care quality is all but impossible to assess unless source data are used throughout the care processes.

The full potential of data-driven precision medicine can only be reached when data are truly liberated. We therefore recommend that care providers in conjunction with med-tech providers, structure all data at the source using internationally adopted standards for clinical informatics and interoperability.

Natural Language Processing: Although we did not achieve clinical-grade precision in NLP, the field is rapidly evolving and may soon provide tools capable of doing so even for poorly structured input data, given enough data for training [9]. Nevertheless, NLP will likely play a major role in structuring retrospective medical chart data for, e.g., calibration or the creation of predictive algorithms, where the tolerance for (random) misclassification is higher. From our experience, NLP may have some benefits. However, there needs to be careful consideration of the format (and variability) of the source data and the intended use of the NLP-generated structured data set.

Patient Surveys: A significant part of this exploratory study involved the recording of patient perceptions. Not least, given the literature on intervention adoption in healthcare, we felt it essential to be able to compare how patients felt about their treatment. Survey response rates were significantly higher in the ISPM in comparison to the baseline cohort. This is a direct consequence of our attempts to encourage participation and not a result of the implementation of ISPM. Patient-reported data should be an integral part of clinical decision-making and therefore supported by technology, for patients to enhance patient engagement, at the point of care to enable precision medicine, and for secondary use to ensure appropriate generalisability of findings. We therefore recommend that specific resources be devoted to the integration of patient interaction tools with CDS technology.

General

In this prostate cancer study, we have measured the impact of the ISPM technology from multiple operational, clinical, patient, and staff satisfaction perspectives. In order to be able to achieve study goals, we had to evaluate the potential for exploiting standard big data techniques. For our domain (prostate cancer), standard algorithms seem appropriate for inclusion as input to enhanced CDS. We went further, though. Recognising that multiple stakeholders are affected by the possible inclusion of these technologies – not least because patient-affecting decisions are made within a cross-disciplinary setting (the MDT) – we have investigated through quantitative and qualitative instruments the perceived benefits of including these innovations. Our findings in this respect are very encouraging and will provide benefit to related studies within a clinical setting. Just as significantly, though, we have highlighted some of the challenges and potential issues that need to be considered moving forward.

In this chapter, we have presented the findings on the introduction of big data technologies into an enhanced decision support system based on the integration of an existing, commercially available software solution. The data suggest that this approach makes MDTs more efficient and improves the process of decision-making.