Identifying key performance indicators in Supply Chain Quality Management 4.0 using machine learning approach Open Access

In today's rapidly evolving industrial environment with I 4.0 technologies, data analytics has become a critical factor in optimizing the flow of goods, services, and information throughout the supply chain. These technologies empower organizations to personalize customer experiences and reshape competitive landscapes (Nardo et al., 2020). The convergence of advanced I4.0 solutions with SC resilience enhances KPIs, improving customer satisfaction and strengthening the organization's strategic position. Adopting a customer-centric approach has become a strategic imperative for achieving sustainable competitive advantage in today's dynamic business environment.

This dynamic fosters organizational agility and innovation by engaging employees in shared objectives, thereby improving overall performance, operational efficiency, and responsiveness to evolving customer expectations. Within this digital transformation, SCQM 4.0 has emerged as a progressive paradigm that integrates digitalization, QM principles, and supply chain management (SCM) strategies. SCQM encompasses the strategic alignment of quality practices across all SC partners to ensure coordination, integration, and continuous improvement in products and services. It extends traditional QM beyond organizational boundaries, emphasizing collaboration, quality assurance, and joint efforts among suppliers, manufacturers, and customers to achieve superior overall performance (Cogollo-Flórez and Correa-Espinal, 2019; Dai et al., 2012). Furthermore, the SCQM 4.0 approach enables organizations to achieve higher levels of transparency, monitor product quality in real time, reduce operational costs, and ultimately foster superior customer satisfaction (Bergman and Klefsjö, 2010). SCQM 4.0 has emerged as a forward-looking paradigm that synthesizes digitalization, QM principles, and SCM strategies, emphasizing the role of smart technologies in driving performance excellence (Bergman and Klefsjö, 2010).

The integration of these three domains-SCM, QM, and I 4.0-not only facilitates process optimization, operational efficiency, and product/service quality enhancement, but also supports informed decision-making and maximized customer value delivery (Duong Thi Binh et al., 2024). The effectiveness and efficiency of traditional QM approaches have been challenged by rapid technological change and evolving customer expectations (Bergman and Klefsjö, 2010). Accordingly, the evaluation of SCQM performance continues to represent a critical challenge for organizations aiming to fulfill increasingly complex customer expectations in the context of digital transformation (Gunasekaran et al., 2019). Although SCQM has received increasing scholarly attention, most of the existing studies focus on limited thematic areas. These include the development of conceptual frameworks, the identification of evaluation indicators, and the integration of SCQM with broader organizational strategies (Bastas and Liyanage, 2019; Bui et al., 2022; Nguyen et al., 2023a). Prior research has mainly emphasized performance evaluation indicators (Karamouz et al., 2021; Machado et al., 2019; Sharma and Joshi, 2023; Truong and Hara, 2018; Truong et al., 2017) and the integration of SCQM with other functional units and broader organizational strategies (Phan et al., 2019). However, given the accelerated pace of technological innovation and the continuously shifting dynamics of business models and SC operations, the concept of quality itself is evolving (Gunasekaran et al., 2019). This transformation has led to the emergence of new structural and operational paradigms that directly influence the principles and practices of SCQM 4.0. Despite extensive research in the field, a clear gap remains in evaluating SCQM performance within the I 4.0 context.

The primary objective of this study is to identify and prioritize the KPIs of SCQM 4.0 by employing an SLR, expert insights, and ML techniques.

This study contributes to the existing SCQM literature by developing a data-driven, empirical framework that integrates ML analysis with expert-driven insights to identify and prioritize KPIs across three levels of the SC (Upstream, Internal, Downstream). From a theoretical standpoint, the study extends prior conceptual models of SCQM 4.0 into a more quantitative and evidence-based context, bridging the gap between digital transformation and quality management research. From a practical standpoint, it provides actionable guidance for managers seeking to enhance quality integration, transparency, and performance across digitalized SC.

The remainder of this paper is structured as follows. Section 2 reviews the theoretical foundations and prior studies related to SCQM, SC 4.0, and Q 4.0. Section 3 describes the research methodology, including the identification of KPIs for SCQM 4.0 through an SLR and expert validation, followed by analysis using ML algorithms. Section 4 presents the empirical findings, while Section 5 discusses the theoretical contributions and managerial implications. Finally, Section 6 concludes the paper, highlighting its limitations and suggesting directions for future research.

The concept specifically refers to quality assurance processes across all SC stages, including the evaluation and improvement of performance as well as the quality of products and services delivered to customers, to ensure customer satisfaction (Karamouz et al., 2021). It also involves quality monitoring at every stage, from raw material procurement and production processes to product quality assurance and final delivery, ensuring that goods and services meet defined quality standards and supporting the continuous improvement of service and product quality (Ardito et al., 2018).

SCQM involves aligning and integrating quality-related processes to promote continuous improvement of product and service quality throughout the entire supply chain. This approach contributes to increased customer value and satisfaction while reducing operational costs (Machado et al., 2019). Duong Thi Binh et al. (2024) define “SCQM as a collection of QM principles applied across the SC to elevate quality standards”. It is viewed as a set of strategic initiatives that organizations implement to enhance the quality of their products and services through collaboration and integration with suppliers and customers. SCQM represents a shift from a task-oriented approach to a strategy-driven framework in QM (Nguyen et al., 2023b).

Q 4.0 is an emerging concept that leverages advanced technologies such as CPS ^[1], the IoT and cloud computing to enhance the quality of products and services throughout the supply chain. This paradigm has garnered significant attention across organizations (Dai et al., 2012; Tambare et al., 2021). Q 4.0 is closely intertwined with I 4.0, representing two complementary dimensions and aspects of digital transformation aimed at organizational maturity (Mittal et al., 2024).

From another perspective, Q 4.0 emphasizes the integration of digital technologies with QM practices to drive continuous improvement, enable real-time monitoring, and support data-driven decision-making, ultimately leading to superior product quality and enhanced customer satisfaction (Mittal et al., 2024). The convergence of QM and digital technologies to meet stakeholders' expectations across the SC is referred to as SCQM 4.0. This concept involves the application of big data, artificial intelligence tools, and advanced analytics to foster and optimize quality and efficiency throughout the SC (Van Nguyen et al., 2023).

SCQM 4.0 is a comprehensive process that employs digital and emerging technologies to support continuous improvement of products and services and to ensure quality throughout the supply chain. This management approach encompasses data integration, enhanced communication, advanced analytics, risk management, accountability, and transparency. Given the growing importance of modern technologies in transforming traditional quality concepts and their role in shaping intelligent SC, the evaluation and analysis of KPIs in SCQM under the I 4.0 paradigm has become an undeniable necessity. Accordingly, understanding the theoretical frameworks and prior studies related to SCQM 4.0 can offer deeper insights into its multifaceted nature and provide a solid foundation for identifying effective performance indicators that guide organizational efforts. Based on this perspective, the following section presents a comprehensive review of the relevant literature, from which KPIs are extracted. Subsequently, the methodology for data collection and analysis is elaborated.

An initial search was conducted on English-language articles published between 1990 and 2024 in the Scopus database. After retrieval, the articles were screened and compared based on their relevance. The search process was guided by the use of specific keywords in the Scopus database. These keywords included: SCQM 4.0, SC 4.0, Q 4.0, SCM, QM, I 4.0.

The search initially yielded a total of 1,492 articles. Among these, 108 articles lacking a DOI were excluded, reducing the number to 1,384. An additional 58 duplicate records were removed, resulting in 1,326 remaining articles. Subsequently, 482 articles with fewer than 15 citations were excluded-this threshold did not apply to articles published in 2024 and 2025, leaving 844 articles. After a thorough screening process, 472 articles were excluded based on irrelevant or insufficiently informative titles, and another 338 articles were eliminated due to abstracts deemed unrelated or inadequate by the authors. Ultimately, 34 articles were selected for in-depth analysis and indicator extraction. A summary of the article selection process is presented in Figure 1.

The literature review of this study shows that, although numerous studies have separately addressed the dimensions of QM, SC digitalization, or their integration, only eight articles have specifically addressed the intersection of these three areas in the context of SCQM 4.0 (Figure 2). Although many studies have explored the conceptual development, strategic implications, and implementation challenges of SCQM 4.0, relatively few have proposed comprehensive, data-driven frameworks for the systematic identification and prioritization of KPIs across SC levels. This indicates that research in this area is still in its early stages, and a comprehensive, data-driven approach to evaluating SCQM 4.0 performance has received little attention in the literature. This gap underscores the need for methodologies that reflect the complexity of modern SC and support intelligent quality decision-making.

In the continuation of this study, a bibliometric analysis was conducted using R-Studio on the final 34 selected articles. Figure 3 illustrates the distribution of the selected articles in the domains of SCQM 4.0, SCQM, Q 4.0, and SC4.0 over the period from 2011 to 2024.

Figure 4 illustrates the distribution of the reviewed articles based on reputable journals with international indexing across the four domains of SCQM, SCQM 4.0, Q 4.0, and SC4.0.

Based on the conceptual frameworks established by Foster (2008), Zeng et al. (2013) and Phan et al. (2019), SCQM can be analyzed across three dimensions: “Upstream”, “Internal”, and “Downstream”. Accordingly, this study identifies and classifies the performance indicators of SCQM 4.0 within these three dimensions. These indicators span various levels of internal processes and inter-organizational interactions. Categorizing KPIs within the upstream, internal, and downstream domains enables organizations to better assess their strengths and weaknesses, thereby facilitating comprehensive performance improvements across all operational levels (Karamouz et al., 2021; Phan et al., 2019; Rasool et al., 2022).

By reviewing the literature, it is concluded that I 4.0 technologies will bring many changes in the future of SCQM. Combination of traditional process with process-centered approaches to digitally enabled, intelligence-driven paradigms under the umbrella of SCQM 4.0. While classical SCQM frameworks focus on integrating quality objectives across the supply chain, SCQM 4.0 extends this foundation through advanced technologies such as IoT, AI, and big data analytics to enhance agility, transparency, and responsiveness.

In this study, based on SLR and expert insights, performance indicators relevant to SCQM 4.0 were identified and categorized. The SLR is used to develop a structured analytical framework. Accordingly, by analyzing the content of 34 articles, activities related to different dimensions of SCQM 4.0 were identified and extracted. After that, a comprehensive set of performance indicators associated with each activity was compiled. This work enables a clear mapping between the indicators and the upstream, internal, and downstream dimensions of SCQM 4.0 (Table 1).

This study adopts a developmental and applied orientation in terms of its objectives and has been adopted in two main phases. The first phase, an SLR combined with expert interviews, was used to determine KPIs for SCQM 4.0. The second phase focuses on prioritizing these indicators using ML algorithms.

In the initial phase, relevant performance indicators were extracted based on an in-depth review of theoretical foundations, prior empirical studies, and expert interviews. This combined approach ensured that the selected KPIs were both theoretically grounded and practically validated. These indicators formed the conceptual basis of the current study and provided the basis for identifying the most relevant KPIs. The target population consisted of professionals and managers, including those in sectors such as manufacturing, electronics, automotive, food, oil and gas, logistics, pharmaceuticals, telecommunications, healthcare, textiles, and education who possess a working knowledge of QM in SC and I 4.0.

A simple random sampling technique was employed. The sample size was determined using Cochran's formula for populations of an unknown or infinite size, with a 5% margin of error and a 95% confidence level. Accordingly, a minimum of 300 participants was required, and a total of 331 valid responses were collected through the survey instrument administered in this study.

This study utilized a questionnaire as a data collection tool. A structured questionnaire, consisting of 62 questions–organized into three sections related to the upstream, internal, and downstream dimensions of SCQM–was distributed electronically among eligible respondents. The questionnaire for this study was designed based on indicators identified from a literature review related to SCM 4.0. To evaluate the validity of the questions, the opinions of professors, experts, and specialists in this field were consulted, and their reliability was assessed using Cronbach's alpha coefficient, which was found to be 0.79. Since the alpha value exceeded 0.7 at this stage, the questionnaire was deemed reliable (Witten et al., 2025). The questionnaire was structured into three sections, consisting of 58 specialized questions and 4 general questions. The questionnaire was designed using a 10-point Likert scale (from very low to very high).

In the next phase, the necessary data for training and testing the models was prepared. The data preprocessing process, including cleaning, normalization, and dataset preparation, was then conducted. Following this, the data were divided into two parts: training and testing, to independently evaluate the performance of the models. The algorithm coding was written in MATLAB software, and the results were extracted for analysis. To accurately evaluate the performance of the ML algorithms, a ten-fold cross-validation method was employed. To select the best algorithm from the ML algorithms used (SVM1 vs All, KNN, SVM, RF, LDA, DT, and RUSBoost), the accuracy metric was used. The results of the algorithms were compared, and the RF algorithm was chosen as the optimal algorithm for identifying KPIs, which served as the basis for analysis in this study. Supervised ML algorithms were used to prioritize KPIs. These algorithms were evaluated under two separate scenarios, each designed to examine the model's performance in different data structures and conditions. As shown in Stage 6 of Figure 5, two analytical scenarios were designed to evaluate the performance of the ML algorithms and identify the most influential KPIs under different data configurations. The goal of two scenarios was to identify the most effective and efficient KPIs for SCQM 4.0 by evaluating the accuracy and efficiency of the algorithm with diverse feature sets and data configurations. A detailed description of two scenarios is provided in the following sections.

In this scenario, all identified indicators were treated as independent variables to provide a holistic assessment of SCQM performance. The overall performance of SCQM within the I 4.0 context served as the dependent variable. This approach allowed the evaluation of how each indicator collectively contributes to measuring digitalized SC quality. By capturing the interrelationships among all variables, Scenario 1 offers an integrated perspective on SCQM 4.0 performance and establishes a comprehensive baseline for subsequent level-specific analyses.

The second scenario examines the indicators within each tier individually, analyzing each level (upstream, internal, and downstream) separately. In this approach, the indicators within each tier are treated as independent variables, and their suitability for assessing QM performance across different levels of the SC in the context of I 4.0 is evaluated. To achieve this, separate supervised ML models were developed for each SC level to capture their distinct performance dynamics. The RF algorithm was chosen because it provided the best evaluation. A 10-fold cross-validation procedure ensured the reliability and generalizability of the models. This scenario provided deeper, data-driven insights into quality performance at each level, enabling more accurate KPI prioritization and supporting informed managerial decision-making within the SCQM 4.0 framework.

By employing the two aforementioned scenarios, a comprehensive analysis is enabled, from the most macro-level perspective to the most granular. In addition to identifying the most effective ML algorithm, the most influential indicators within each tier are also determined. Following the evaluation of the models developed under each scenario, the algorithms are compared based on the Accuracy metric. The algorithm that yields the most reliable results serves as the basis for selecting the KPIs for SCQM 4.0.

The proposed model must possess the capability to predict future observations; therefore, it should not rely solely on the training dataset but must demonstrate generalizability and scalability for real-world applications (Witten et al., 2025). In classification problems, the most critical evaluation metric is Accuracy, which reflects the proportion of correctly classified instances to the total number of instances. To comprehensively evaluate the model's performance, classical classification evaluation criteria, including overall Precision, Recall, F1 Score (the average of precision and recall), Accuracy, and Confusion Matrix, were used. Finally, Accuracy was chosen as the evaluation criterion, and the analyses of this study were conducted based on this criterion. The accuracy criterion is denoted by Acc(X):

In this study, the dataset was split into three subsets: training, validation, and testing. This strategy enhances the realism of performance prediction by maintaining independence among datasets, thereby reducing the risk of overfitting and improving the generalizability of the model. In addition, the k-fold cross-validation method with k = 10 was employed. K-fold is a method to ensure robust and reliable model evaluation (Borovicka et al., 2012).

Following the completion of questionnaires by respondents, a total of 331 valid data entries were collected. Data preprocessing steps, including overfitting control and normalization, were performed. Subsequently, the data were analyzed under two distinct scenarios using 7 ML algorithms implemented in MATLAB. Model performance was evaluated using 10-fold cross-validation and the Accuracy metric. Feature selection techniques were applied to identify influential variables and enhance model performance. To assess reliability, Cronbach's alpha was calculated for all validated questionnaires, yielding a value of 0.79.

As illustrated in Figure 6, the respondents represent a diverse range of industries, including manufacturing, automotive, food, logistics, electrical and electronics, textile, healthcare and pharmaceuticals, telecommunications, oil and gas, and education. This diversity ensures that the identified KPIs for SCQM 4.0 reflect a broad cross-section of industrial practices, thereby enhancing the generalizability of the study's findings.

This section shows the results of applying each of the seven algorithms across different scenarios, to identify the KPIs.

The results of the performance comparison of the ML algorithms examined in this study, based on the accuracy metric, are presented in Figure 7. As illustrated, the RF algorithm achieved the highest accuracy score (0.81) among all evaluated models.

As shown in Figure 8, 6 ML algorithms -“including RF, SVM, SVM 1-vs-All, KNN, LDA, and DT”- were evaluated during the feature selection phase to determine the most effective algorithm for identifying KPIs of SCQM 4.0. Although LDA and SVM initially exhibited relatively high accuracy scores (approximately 0.81), their performance was strongly dependent on specific assumptions, such as data normality and homogeneity of variance across classes. These conditions are rarely satisfied in real-world, non-linear, and heterogeneous datasets, leading to significant performance degradation.

In contrast, the RF algorithm consistently demonstrated more stable and reliable performance while maintaining a comparably high level of accuracy. Its robustness to varying feature counts and its ability to capture complex non-linear relationships make it particularly well-suited for analyzing the multifaceted data associated with SCQM 4.0. In contrast, compared to algorithms such as SVM, DT, KNN, and SVM 1-vs-All, which experienced notable fluctuations or performance drops at various stages, RF achieved a superior balance between accuracy, stability, and generalizability. As shown in Figures 7 and 8, RF not only achieved the highest classification accuracy but also provided a more balanced trade-off between precision, stability, and generalizability. These advantages position RF as the most appropriate and reliable algorithm for identifying and prioritizing KPIs within this study.

As illustrated in Figure 8, the RF algorithm exhibited more consistent and accurate performance compared to the other algorithms in classifying the KPIs of SCQM 4.0. It therefore stands out as the most appropriate and effective model in this context. The performance of the RF algorithm in the feature selection stages for the top 10 key indicators is visualized in Figure 9. The horizontal axis represents the feature selection stages, labeled with corresponding indicators (e.g. Q25, Q3, Q50, etc.), while the vertical axis displays the model's accuracy. The numerical values above each bar indicate the exact accuracy achieved for each indicator.

As shown in Figure 9, indicators Global Connectivity (Q₂₅) and Supplier Responsiveness (Q₁) demonstrated the highest classification accuracy among all features, both achieving an accuracy score of 0.81. These were followed by Participation of Customers in Quality Control (Q₅₅) and Responsiveness to Customers/Customer Response Handling (Q₄₉), each with an accuracy of 0.80. Additional top-performing indicators include Participation of Suppliers in Quality Controls (Q₁₀), Supplier Resilience (Q₀₂), Intelligence Supply/Information Provision (Q₃₆), Customer Satisfaction (Q₅₀), Supplier Flexibility and Agility (Q₀₃), and Customer Complaint (Q₅₁). The features selected at this stage have the highest predictive power among other indicators and are interactive. Digitalization, and related to SC responsiveness.

The results of this section reveal that SCM performance in I 4.0 era is not solely dependent on Internal organization production processes. Through the identification of influential indicators, the RF algorithm underscores this reality. The prioritization of indicators in this scenario highlights that QM in the digital age is profoundly linked to communicative capabilities, digital resilience, data-driven insights, and customer understanding.

In this scenario, to determine the optimal algorithm, a comparative analysis was conducted across the three levels of the supply chain-upstream, internal, and downstream. As shown in Figure 10, the RF algorithm consistently achieved the highest accuracy across all levels, outperforming the other models. This result indicates that RF not only possesses strong capabilities in modeling complex relationships but also demonstrates adaptability to diverse indicators at each layer of the supply chain. Moreover, due to its ensemble structure, which integrates multiple DTs, RF is inherently less prone to overfitting. The algorithm's superior performance can also be attributed to its robustness in handling high-dimensional and heterogeneous data. While algorithms such as KNN, SVM1-vs-All, and DT showed relatively good performance at certain levels, models like RUSBoost exhibited low accuracy across all levels and failed to deliver satisfactory results. Overall, given its consistently high accuracy and strong generalization ability, Random Forest is selected as the most reliable and powerful algorithm for prioritizing KPIs at each level of SCQM 4.0.

Subsequently, the accuracy of the RF algorithm was examined across different feature selection stages within the SC levels. Based on this analysis, the indicators were evaluated and prioritized separately for each tier of the supply chain. This prioritization provides a solid foundation for selecting and focusing on the indicators that exert the greatest influence on performance improvement across the various levels of the supply chain.

The accuracy of the RF algorithm at the upstream level, along with the analysis of the top 10 key indicators in this tier, is presented in Figure 11 and Table 2. As depicted, the dimensions related to technology and information sharing, supplier competence, and supplier involvement have the highest impact on SCQM 4.0 performance. The indicators associated with these dimensions include Sharing Information About Cost, Benefits, and Quality With Suppliers (Q₀₇), Supplier Responsiveness (Q₀₁), Buyer–Supplier Partnership Level (Q₁₂), Supplier Flexibility and Agility (Q₀₃), Integrating Supplier Capabilities Into the Design Process (Q₀₈), Rejection Rate/Rate of Supplier Product Rejection (Q₀₅), and Supplier Social Responsibility (Q₀₄) rank highest in importance.

Figure 11 indicates that the accuracy of the RF algorithm in analyzing the indicators at the upstream level ranges between 0.54 and 0.66, reflecting an acceptable level of model stability and strong discriminatory power among the key indicators at this tier. These findings suggest that within the SCQM 4.0 framework, the quality of supplier selection should not be viewed solely from a product-based perspective. Rather, it encompasses the utilization of technological infrastructure, the selection of competent suppliers, and Cooperation and information sharing. When these factors are aligned, they can play a critical role in enhancing quality and reducing risk across the supply chain. The high importance of these indicators demonstrates the importance of rapid response, adaptation to changes, and active supplier participation in technical and quality decision-making.

Figure 12 presents the analysis of indicator importance at the internal level using the RF algorithm. The most influential SC dimensions at this tier include digital value creation, strategic management and planning, digital supply network infrastructure, and cost management, exerting a significant impact on SCQM 4.0 performance. As shown in the figure, the accuracy scores associated with these features range from 0.51 to 0.60, indicating the model's strong discriminative ability and the quality of distinction among the selected indicators. These findings confirm that, at the internal level, the integration of technological innovation, data-driven decision-making, and workforce empowerment forms a critical foundation for the successful implementation of SCQM 4.0.

Prioritization of key indicators at the internal level is presented in Table 3. As illustrated in Table 3, indicators such as Intelligence Supply/Information Provision (Q₃₆), System Ease of Use/System Usability (Q₃₈), Workplace Safety (Q₃₀), Digital Collaboration (Q₃₁), and Information Transparency (Q₁₆) play a decisive role in enhancing internal quality performance.

These indicators significantly contribute to the accuracy and stability of SCQM 4.0 performance. Moreover, the presence of indicators, such as Product Design and Optimization Processes, Digital Innovation, and Coordinated Planning and Execution, among the subsequent priorities demonstrates that effective SCQM 4.0 implementation requires strategic digital alignment within the organization. Managers should place their focus on indicators that are data-driven, traceable, and intelligent. Such focus can enhance the effectiveness of decision-making, strengthen internal coordination, and improve organizational agility in the digital era.

At the downstream level of the supply chain, data analysis was conducted using the RF algorithm (Figure 13). The accuracy values across the indicators ranged from 0.43 to 0.62. This variation reflects the strong capability of the RF model in prioritizing the selected indicators at the downstream tier. As shown in the prioritized results presented in Table 4, customer engagement, whether through traditional channels or digital technologies, along with active participation and a deep understanding of customer needs, plays a fundamental (key) role in the performance of SCQM 4.0.

As further shown in Table 4, the most influential factors contributing to quality enhancement at the downstream level of the SC include Product Delivery Reliability (Q₄₈), Digital engagement and Top line growth (Q₅₇), Customer Responsiveness (Q₄₉), On-Time Delivery (Q₅₂), and Customer Satisfaction (Q₅₀). Additionally, information sharing with customers (e.g. regarding cost and quality), understanding customer needs, and integrating their capabilities into process design serve as foundational elements for quality improvement at this tier. The RF algorithm strongly affirms that success in SCQM 4.0 cannot be achieved solely through production optimization; rather, it requires a deliberate focus on customer engagement, feedback analysis, and digital synergy through continuous interaction and insight into customer expectations. Overall, achieving excellence in this domain demands a strategic blend of technological capability, deep customer insight, and the development of responsive, transparent systems for ongoing customer interaction. This approach enhances customer satisfaction, increases traceability, continuously improves product quality, and increases market share.

As shown in this section, the RF algorithm showed the highest prediction accuracy and stability in all scenarios and was selected as an effective algorithm for prioritizing KPIs in the SCQM 4.0 framework. The results obtained from identifying and prioritizing KPIs in SCQM 4.0 using the ML approach are further explained in the next section to provide a summary of the key findings of the study and its practical implications.

Traditional SCQM models no longer meet the complex demands of digitalized supply chains under I 4.0. Achieving next-generation quality, focused on sustainability, agility, and competitiveness, requires leveraging advanced technologies such as IoT, big data, ML, and AI. To address these challenges, this study integrates data-driven methods with managerial theory to identify and prioritize KPIs for SCQM 4.0.

A review of 34 articles showed that only eight explicitly address the intersection of SCM, QM, and I 4.0, collectively defined as SCQM 4.0, and most were conceptual rather than empirical. This gap highlights the need for data-driven approaches to evaluate SCQM performance. The present research fills this gap through an ML-based framework that empirically prioritizes KPIs across the upstream, internal, and downstream dimensions of the supply chain.

Figure 14 presents the hierarchical structure of KPI prioritization, illustrating how key indicators are distributed across the three SC levels and interact to form an integrated performance framework. This visualization clarifies both the relative importance of each KPI and its nesting within the overall SCQM 4.0 system.

At the upstream level, key indicators, supplier responsiveness, flexibility and agility, integration of supplier capabilities, information sharing, and social responsibility, show that supplier capability and transparency underpin supply-chain quality. These findings are consistent with Zhang et al. (2021) and Srinivasan and Swink (2018), but extend prior work by emphasizing digital coordination and information sharing as critical enablers of upstream quality. Similarly, (Bastas and Liyanage, 2019; Bui et al., 2022; Nguyen et al., 2023a) emphasized the role of supplier collaboration and integration as critical enablers of SCQM performance. However, the present study extends their conceptual arguments by providing empirical evidence, through ML-based modeling, that supplier responsiveness and digital information sharing platforms directly enhance upstream quality resilience under the SCQM 4.0 paradigm.

At the internal level, indicators such as digital innovation, continuous improvement, information transparency, and synchronized planning exert the strongest influence. Aligned by Ivanov and Das (2020) and Zamani et al. (2023) these results confirm that robust digital infrastructure and a learning-oriented culture drive SCQM 4.0 success. Moreover, this study extends prior models Foster (2008) and Zeng et al. (2013) by revealing that internal integration and process digitalization serve as the connecting mechanism between technological capability and operational efficiency.

At the downstream level, digital interaction, responsiveness, and customer participation dominate. Indicators such as complaint handling, satisfaction, and product customization shape perceived value and loyalty. Consistent with Kache and Seuring (2017) and Dubey et al. (2021), these findings stress customer-centric practices, but this study further demonstrates, through data-driven validation, that customer involvement and digital co-creation enhance performance sustainability in smart supply chains.

Overall, the comparative analysis confirms that SCQM 4.0 performance results from the interplay between supplier collaboration, internal digital capability, and customer integration, consistent with previous studies emphasizing multi-tier interconnectivity in digital SC (Ivanov and Das (2020), Zhang et al. (2021)). Combining SLR and ML analysis enhances analytical precision and provides a data-driven, yet practically relevant, understanding of how digital quality evolves in interconnected supply chains.

Although the level-specific accuracy analysis was not detailed in the Findings section, complementary testing confirmed that the Random Forest algorithm maintained stable and high predictive accuracy across all three tiers: Upstream (0.83), Internal (0.95), and Downstream (0.71). These additional insights reinforce the robustness and generalizability of the model. The comparative assessment across the upstream, internal, and downstream levels reveals mutually reinforcing relationships, indicating that these dimensions collectively shape overall SCQM 4.0 performance. The strong predictive results observed at the internal level highlight its pivotal role as a bridge connecting supplier-driven and customer-oriented quality improvements. Overall, the findings suggest that QM in digitalized supply chains operates as an integrated system where upstream collaboration, internal capabilities, and downstream responsiveness continuously strengthen and support each other. Moreover, although each tier (Upstream, Internal, and Downstream) exhibits analytical independence in terms of data and KPI assessment, the findings highlight a strong functional interdependence among them. The internal tier acts as a mediating bridge, linking supplier inputs with customer outcomes. This demonstrates that SCQM 4.0 functions as an integrated, adaptive network where improvements in one tier enhance performance across the others.

Compared to the study by Dubey et al. (2021), which primarily focused on resilience and sustainability within digital supply chains, the present research adopts a more comprehensive approach that integrates social, environmental, economic, and technological dimensions. Similarly, Bastas and Liyanage (2019) highlighted the need to integrate quality and SC practices for sustainable competitiveness; this study extends their conceptual insights by empirically validating these relationships under the I 4.0 context. The inclusion of indicators such as supplier and customer participation, organizational agility and flexibility, and organizational learning suggests a paradigm shift from a purely technological orientation toward a broader understanding of behavioral and cultural dynamics in SCQM. This shift is particularly vital in the context of Q 4.0, where rapidly changing environments require adaptive, learning-based quality frameworks.

These findings are consistent with previous works, including Foster (2008) and Zeng et al. (2013), which emphasizes that SC quality is contingent upon transparent information flows and active stakeholder engagement. However, this study extends prior theory by applying ML algorithms, particularly RF, to reveal nonlinear and multi-level relationships among KPIs across upstream, internal, and downstream dimensions. This methodological innovation deepens the theoretical understanding of how data-driven approaches transform SCQM from static evaluation models into dynamic, intelligent systems.

The results also align with and expand upon Bui et al. (2022), who emphasized digital integration as a foundation for SCQM 4.0 maturity, while this study operationalizes those concepts through ML-based empirical modeling. In essence, this research helps to improve the SCQM 4.0 process by integrating digital, behavioral, and organizational dimensions into a single framework. This research empirically supports the notion that intelligent algorithms can link conceptual models with operational realities; while simultaneously advancing theory and methodology in this field.

The analysis conducted in this study, particularly using ML algorithms, assists managers in making data-driven and targeted decisions for quality improvement and performance optimization. The hierarchical classification of KPIs enables managers to identify the most critical indicators at each SC level and to align digital initiatives accordingly.

Practically, this research guides organizations in integrating ML-based evaluation tools into their QM systems, moving from reactive control to predictive and adaptive decision-making. The identified indicators can support strategic planning, supplier collaboration, and customer engagement within digital ecosystems.

Furthermore, the study's results suggest that developing intelligent, AI-based tools for KPI monitoring could enhance the practical implementation of SCQM 4.0 in real-world industrial environments. This approach empowers organizations to strengthen competitiveness, enhance sustainability, and sustain long-term customer satisfaction.

This study aimed to identify and prioritize the KPIs influencing SCQM 4.0 by applying and comparing seven supervised ML algorithms across three distinct SC levels: Upstream, Internal, and Downstream. Among the evaluated models, the RF algorithm achieved the best overall performance, demonstrating high accuracy, robustness, and the ability to capture data. Based on these findings, the following subsections summarize the main results, theoretical and practical contributions, implications of the research, limitations, and future research. At the upstream level, the RF algorithm identified “use of information technology for supplier communication and process optimization,” “supplier flexibility,” “supplier product nonconformance rate,” “supplier motivation,” and “social responsibility” as the most influential indicators. These factors play a critical role in ensuring input quality, strengthening collaboration, and improving resilience across inter-organizational relationships.

At the internal level, the most significant indicators were “digital innovation,” “continuous improvement and learning,” “coordinated planning and execution,” “total cost of ownership control,” and “product and process design optimization.” These indicators act as key enablers for enhancing internal quality, efficiency, and operational integration within SCQM 4.0.

At the downstream level, “customer complaint management,” “technology-driven product development,” “integration of customer capabilities into process design,” “customer satisfaction and needs recognition,” and “information sharing with customers” emerged as critical indicators. These highlight the importance of intelligent, two-way communication with customers to enhance satisfaction, responsiveness, and product quality. Overall, the results indicate that the upstream, internal, and downstream dimensions of SCQM 4.0 complement one another, with internal capabilities serving as a connecting element that strengthens both supplier integration and customer responsiveness across the SC.

This study enriches the existing SCQM 4.0 literature by integrating ML-based analytics into QM models. It extends prior conceptual frameworks by demonstrating how ML techniques, particularly RF, can identify nonlinear, multi-level relationships among KPIs. This approach transforms SCQM from a static evaluation system into a dynamic, data-driven decision framework, contributing to the theoretical advancement of digital QM. From a managerial perspective, the proposed framework provides a structured and evidence-based tool for KPI prioritization at each SC level. It supports strategic alignment among supplier collaboration, internal process optimization, and customer engagement under I 4.0 technologies. The results enable organizations to shift from reactive quality control toward predictive and adaptive management, thereby enhancing competitiveness, agility, and long-term sustainability across the supply chain. This study has several limitations. First, it relied on survey-based data reflecting perceptions rather than real-time operational data, which may limit the precision of behavioral insights. Second, although several ML algorithms were compared, hybrid or deep learning models were not examined. Third, the analysis covered multiple industries without distinguishing sector-specific variations in KPI importance. Additionally, the focus on accuracy as the primary performance metric may have constrained the comprehensiveness of algorithm evaluation.

Future research can build on this work by applying the framework in specific industries (e.g. automotive, healthcare, or energy) to capture contextual variations. Incorporating real-time IoT or ERP data could improve the responsiveness of KPI monitoring systems. Testing advanced algorithms such as XGBoost, LightGBM, or deep learning models may enhance predictive performance. Combining ML with statistical or structural approaches like Structural Equation Modeling (SEM) could yield deeper analytical insights. Finally, future studies should explore integrating quality and sustainability dimensions to support the transition toward SCQM 5.0 frameworks.