Assessment instrument for lean journeys in integrated project delivery: lean learning health assessment (LLHA) Open Access

The architecture, engineering and construction (AEC) industry has significant potential for improvement. This growth can be realized by capitalizing on the capabilities of Lean in both design and construction (Dodge Analytics, 2021). However, the industry's growth potential is hamstrung by waste, inefficiencies and a lack of collaboration, practices associated with traditional approaches to design and construction. Currently, 80% of construction projects are delivered over budget and take 20% longer to be completed (McKinsey, 2020). These prevalent challenges demonstrate the need for more efficient and streamlined planning and execution of projects.

The clear need for improvements has driven organizations, researchers and industry practitioners to pursue the use of Lean (Dodge Analytics, 2021) efforts aimed at assessing the application of integrated Lean methods to bolster the design and construction process outcomes (Ahmed and El-Sayegh, 2020; Vaagen and Ballard, 2021). The benefits derived from the Lean initiatives help deliver projects under budget, on schedule and with highly integrated and competent teams (Ling et al., 2020). Integrated project delivery (IPD) provides a more streamlined environment for incorporating Lean in projects (Choi et al., 2019), although Lean principles can be used with any form of collaborative project delivery methods.

Recent practice has further explored opportunities for enhancing the performance recorded by IPD projects by embedding additional process-oriented production improvement methods, such as Building information Modeling (BIM) (Bhargav and Sacks, 2020). Both BIM and Lean support the required level of collaboration intensity in IPD projects, allowing project teams to develop the design and production process at an optimum level (Herrera et al., 2021). The nexus of BIM and IPD has been extensively discussed in literature (Kahvandi et al., 2017). However, the level of understanding and implementation of Lean in integrated projects is yet to be fully explored despite documented benefits associated with Lean in IPD contexts (University of Minnesota, 2016).

In integrating Lean and IPD, the most profound barriers to successful integration stem from participants' conversance with Lean and IPD concepts (Evans et al., 2023). There is a need for industry professionals to build a strong awareness of Lean to achieve the much-needed progress toward Lean implementation in IPD projects. Participants in these projects should understand Lean principles, methods and tools and mechanisms of identifying the challenging areas when applying Lean methods. This understanding can be imparted through workshops during the onboarding process and continuous training. The onetime and progressive trainings support continuous learning of Lean participants (Messner et al., 2019). However, there are currently limited frameworks to support structured learning processes at an individual level in Lean-driven IPD projects. Coupled with the intensity of the IPD , practitioners may lack the time to continuously participate in extensive training, gauge their own knowledge on Lean principles and track the gaps in implementing Lean concepts on their own (Seed, 2014).

To bridge this gap, this paper proposes and validates an instrument for assessing the Lean knowledge areas among IPD participants based on self-regulated learning theory. This assessment instrument aims to identify areas of strength and opportunities for subsequent knowledge enhancement for these Lean-IPD project participants. The paper discusses the development and validation of the self-assessment instrument applicable to IPD projects. The practical aim of the instrument is to allow practitioners to gain awareness of their Lean journey based on the outcomes of their responses. The tool was tested on 123 participants from three IPD projects within the US construction industry. Through a principal component analysis (PCA), the data from the self-assessment were analyzed to determine the extent to which different participants have different awareness and levels of emphasis on various Lean principles.

Over recent decades, the construction industry has lagged behind in productivity, quality and efficiency compared to other economic sectors such as manufacturing and service industries, which have demonstrated remarkable improvements (McKinsey, 2020). These growth areas of productivity and efficiency have stagnated, yet escalation in cost and schedules have risen higher than expected (Franz et al., 2020). One of the primary causes of inefficiencies resulting in these escalations is the fragmented nature of the industry disciplines, processes and methods of delivery of construction projects (Levitt and Plambeck, 2010).

The poor performance is associated with the silos in communication and collaboration during the design and construction processes (Franz and Roberts, 2022). These silos are characteristic of the most prevalent delivery method, Design-Bid-Build, which has been criticized for resulting in a significant amount of non-value adding efforts (Ahmed and El-Sayegh, 2020). These conventional project delivery methods exhibit lower performance outcomes due to the higher level of fragmentation of both technical and social processes and participants (Franz et al., 2020; Sullivan et al., 2017).

The lack of sufficient integration limits opportunities for the designer, constructor and the client from having a common understanding of the execution strategy of the project. The outcome is often exhibited in the low quality or productivity issues experienced in downstream production processes (Dodge Construction Network, 2024). The industry has brought forward alternative methods of project delivery, implemented alongside Lean methods to address these challenges. Integrated Project Delivery (IPD) is among the most recent transformations. IPD uses multi-party relational contracting to enhance collaboration among project participants and provide financial incentives to drive value and implement innovative solutions in the project (Kahvandi et al., 2017; Ling et al., 2020).

According to the American Institute of Architects (AIA), IPD was developed to improve the process and practices in the construction of projects to generate value for the owner throughout the facility lifecycle (AIA, 2012). IPD consolidates systems, practices, people and business structures within the construction project domain. To further enable the integration and generate better outcomes, IPD leverages Lean principles and techniques (Ling et al., 2020). Lean prioritize value-add by eliminating waste or non-value-adding activities (Vaagen and Ballard, 2021). Lean champions for efficiency by driving operations as a whole rather than a series of independent activities. This effort is enabled by a range of principles, methods and tools. Projects implementing Lean have recorded improvements of up to 84% in quality, shorter construction durations, higher productivity and up to 77% improvement in safety (Cassino et al., 2013; Dodge Analytics, 2017, 2021). IPD, on the other hand, provides a platform for alignment of interest among participants, enabling a better collaboration environment among the owner, designers, contractors and trade partners (AIA, 2007; Ling et al., 2020).

Incorporating Lean into IPD projects allows the team to tap into the advantages of integration of participants to fully improve efficiency and enhance value. Research exploring the integration of Lean and IPD emphasizes the application of Lean methods and tools to influence the performance outcomes of these projects (Nguyen and Akhavian, 2019). Other researchers have focused on how Lean tools support collaboration among project participants in IPD projects (Fakhimi et al., 2016). However, studies on the level of understanding of the Lean methods and tools among IPD project participants remain low despite the low level of awareness on Lean among practitioners (Cassino et al., 2013). To successfully achieve the desired benefits of Lean, IPD participants must gain knowledge and understanding of strategies for capitalizing on Lean to deliver improved outcomes (Messner et al., 2019).

A transformational change is required to support implementing new project delivery models, like IPD, in an industry dominated by traditional, siloed project delivery practices. This shift demands a new type of project leadership with a knowledgeable and adaptable project manager, otherwise known as an Integrated Project Manager (IPM). This project manager needs to be involved from the early stages of the project. The shift also requires traditionally trained project managers to develop a relationship-based management style that emphasizes continuous improvement, training and guiding the team toward innovation (Seed, 2014).

The IPD delivery process requires the IPM to create a collaborative environment where architects, owners and builders can share ideas and work collaboratively on the project (Vaagen and Ballard, 2021). Therefore, the IPM can help reduce fragmentation normally encountered early on in the project design phase. IPD also requires a project manager who can leverage the team members' abilities by knowing their unique strengths to build a team mentality among all participants (Franz and Roberts, 2022). The IPM should aim at transforming the project organization from a centrally controlled to an evenly spread execution structure (Herrera et al., 2021). Future and current generations of project managers must acquire a new skill set, which aims at developing and leading integrated project teams. However, to acquire these new sets of skills, a project manager must be able to assess their current skills and capabilities to identify potential areas of growth.

Focusing on the existing problem of lack of a Lean learning assessment framework, the study adopts a pragmatic approach combining multiple methods to achieve the study objective (Creswell and Poth, 2017). The study sought to develop an assessment instrument to support individual awareness of Lean learning efforts among IPD project participants. The instrument can help gauge the level of understanding among IPD participants pursuing self-regulated learning of Lean principles, methods and tools. The study was conducted in three parts: instrument development, testing and validation with real-world project participants from three IPD project teams. The first part of the study utilized existing literature on self-regulated learning and assessment methods to develop the Lean Learning instrument. The second step involved the evaluation of the face and content validity of the instrument through interviews with subject matter experts. The last step included validation of the instrument through responses from participants in the IPD three case study projects.

The study sought to learn from and validate the instrument using projects that fit the IPD category. The projects selected also needed to be actively implementing Lean principles, with participants using different Lean tools and methods to achieve the Lean goals and integration of processes and participants in line with IPD principles. The three projects were therefore selected based on alignment with these specific requirements to study projects implementing IPD and Lean in the project processes. The projects were drawn from networks within the Lean construction community in the USA and verified for appropriateness for this study before inclusion as a case study project. The verification process determined alignment with typical practices in Lean-IPD projects in this context, primarily the integration of parties and process using a single contract and the use of Lean principles, methods and tools. Data collected from the participants were analyzed through a PCA. Figure 1 summarizes the steps involved in the study.

The construction industry is taking steps to overcome the issues and challenges arising from the existing siloed approach to project execution. One method to achieve such a goal is by shifting towards IPD in project delivery. The new approach requires project managers to acquire Lean knowledge and integration skills (Evans et al., 2023). Such transformation can be supported by assessment tools that measure both social and technical understanding of Lean and IPD concepts. However, existing instruments do not provide metacognitive awareness of one's Lean journey. In this paper, the authors propose a new assessment instrument designed to support professionals in assessing their Lean journey. This instrument, named the Lean Learning Health Assessment (LLHA), provides a metacognitive awareness and regulating skills of project participants in a wide range of Lean/IPD principles and techniques.

This instrument was developed through a critical analysis of literature on knowledge assessment methods and self-regulated learning theories, to support individuals who are not participating in an educational program and desire to self-regulate their learning. Therefore, the tool's purpose was to assist individuals in assessing their journey in learning Lean principles and methods, to direct and regulate their learning efforts. The intent of this assessment is for someone to determine their knowledge, confidence and motivation in learning and applying techniques, effort regulation and performance of techniques. From the results, individuals will be able to discover their current areas of understanding and potential areas of growth to support their journey towards continuous improvement.

To support assessment of capabilities, team members can leverage various tools with the potential to evaluate strengths, self-behavior and interaction with others and support integration within a team. Some of the more common assessment instruments provide pathways to support the identification of potential areas of improvement, as summarized in Table 1.

These tools are instruments for project leaders to assess themselves and their teams' strengths and weaknesses. By performing these assessments, an individual can further their journey toward becoming an effective team member, leader and manager. Researchers have evaluated the effectiveness of these tools in various scenarios and illustrated their benefits in the process of forming a team (Macht et al. 2011, 2013; Leicht et al., 2012). However, if a project manager wanted to start their journey in Lean construction and become an IPM or a Lean construction expert, these tools are not designed to identify the areas of growth needed. Similarly, none of these tools can support a project participant in assessing their current knowledge or developing an awareness of their knowledge on Lean principles.

Self-Regulated Learning (SRL) is largely defined as the learning process guided by strategies, metacognition and motivation (Schunk and Zimmerman, 1994; Winne and Perry, 2000; Schunk, 2011). By implementing SRL training, researchers have illustrated advancement in the areas of concentration, procrastination, self-efficacy and motivation (Schmitz and Wiese, 2006; Zimmerman, 2008). SRL focuses on the areas of cognition, motivation, behavior and context (Pintrich, 2004). The first three domains specify the knowledge gained, whereas context represents the usefulness and value in social interactions. Each of these domains can be scaled. Cognition measures organization, critical thinking and metacognition. Metacognition refers to the understanding of one's own strengths and weaknesses and creating an awareness of how to regulate performance and knowledge of the task (Winne and Perry, 2000; Schunk, 2011; Schunk et al., 2012). Regulation of cognition suggests learners follow different strategies for planning goals and forethought activities that can include reading the materials beforehand or defining a target to achieve. The next step is monitoring the progress to find the deviation from the goals and finally controlling and adapting the metacognitive activities to bridge the gap by regulating learning (Winne and Perry, 2000). Regulation of motivation determines an individual's mastery over the deep learning of a task (intrinsic) and surface learning from external sources (extrinsic). Self-regulation from context includes the circumstances occurring during the learning process that are not under the control of an individual (Pintrich, 2004).

Measurement of SRL can be achieved using various techniques. Questionnaires and interviews are widely used methods to measure SRL (Winne and Perry, 2000). Some commonly used questionnaires are the Learning and Study Strategies Inventory (LASSI) and the Motivated Strategies for Learning Questionnaire (MSLQ) (Weinstein et al. 1988, 2000; Pintrich and Groot, 1990; Pintrich, 1991; Pintrich et al., 1993). The questions include aspects of organizing, planning, monitoring, environment structuring, seeking help, reviews and self-evaluation. With these results, one can have a view of one's own self-regulation. Pintrich (1991) developed the MSLQ to evaluate one's self-regulatory approach by categorizing the strategies as motivational and learning. According to Pintrich (1991), the learning perspective of SRL is inclusive of motivational, cognitive and social contextual aspects. Another technique to measure self-regulation is the Think-Aloud Protocol that analyzes the reports or thoughts that a student prepares while performing the task. This method is an open-ended methodology in which individuals determine the personal context, their mindset and processes that lead to the effectiveness of the performance of a task (Zimmerman, 2008). These assessment methods for SRL can be leveraged by practitioners in assessing their own knowledge and regulating their learning of Lean construction principles and strategies.

The purpose of the LLHA instrument is to evaluate an individual's motivation, knowledge, effort, performance and awareness in their self-regulated Lean learning journey. To achieve this, the researchers developed a model of the dimensions of self-regulation to be assessed, summarized in Figure 2. This model was based on the four stage model of self-regulated learning developed by Winne and Hadwin (1998) and on the MSLQ scales by Pintrich (1991). The dimensions were based on Pintrich's (2004) self-regulated areas of cognition, motivation, behavior and context. These dimensions were also influenced by Winne and Hadwin's (1998) four-stage model of self-regulated learning. For example, Pintrich's (2004) cognition area of self-regulation was leveraged to develop LLHA's knowledge and strategy dimensions. Similarly, LLHA's motivation dimension was aligned to Pintrich's (2004) motivation area of self-regulation. By assessing these dimensions, an individual can develop a metacognitive awareness of their Lean journey. Lastly, LLHA's dimensions for effort and performance were based on Pintrich's (2004) behavior area, each with a different path as shown in Figure 2.

For each of the dimensions of motivation, knowledge, effort, performance and awareness within an individual, the LLHA instrument assesses the abilities of an individual within four paths of Wisdom, Strategy, Mindfulness and Leadership.

The first dimension that the LLHA assesses motivation, which in this case refers to a process that promotes and enhances active learning, leading to goal achievement (Schunk et al., 2012). Motivation measures various aspects of learning, such as self-efficacy, intrinsic and extrinsic goal orientation, and task value under the categories of self-evaluation and self-consequences (Pintrich, 1991, 2004; Zimmerman, 2008). Motivation is essential in becoming an expert by identifying personal interest and determination. Based on the MSLQ instrument developed by Pintrich et al. (1993), the research team composed a series of questions to determine an individual's motivational level in the four paths. The following are example questions from the assessment instrument:

1. Wisdom. Developing a strong knowledge of Lean principles is valuable for me.

2. Strategy. It is very important for me to master the following Lean management strategies and techniques: The five phases of the Last Planner System.

3. Mindfulness. I believe that performing retrospective activities brings great value; and

4. Leadership. I want that all my team members are able to maintain a collaborative environment based on respect.

The dimension of knowledge was aligned with the “Domain Knowledge,” “Knowledge of Task” and “Knowledge of Study Tactics and Strategies” of the four-stage model of self-regulated learning developed by Winne and Hadwin (1998). The attribute of knowledge specifies that the learner should be aware of the domain, the tasks to perform and the intelligence necessary to complete the activities using various techniques (Winne and Perry, 2000). To deliver a successful Lean/IPD project one should possess a strong foundational knowledge of Lean principles and techniques. Therefore, the research team developed questions evaluating an individual's confidence level in their knowledge within the four paths. The following are example questions from the assessment instrument:

1. Wisdom. I am confident I have master level knowledge of the following concepts: Value Stream Optimization.

2. Strategy. I think I have strong knowledge about the following techniques: Ohno Circles.

3. Mindfulness. I have a strong knowledge of what retrospective activities are; and

4. Leadership. I don't think I can train/teach my team members in: The 5 Principles of Lean Construction.

The effort dimension in LLHA was aligned with the Effort Regulation scale of learning strategies. Pintrich (1991), in his Motivated Strategies for Learning Questionnaire (MSLQ model), considers effort a significant factor of self-management. Effort determines the level of attention paid to complete a particular task. It signifies one's focus towards their commitments to achieve goals and managing through distractions or difficulties. Effort is a valuable aspect of measuring an individual's success as it assesses self-control and self-regulation towards accomplishing targets and continuous learning (Pintrich, 1991). Consistent effort is required to succeed in learning and achieving mastery of Lean principles and techniques. Therefore, the research team developed questions evaluating an individual's effort regulation. The following are example questions from the assessment instrument:

1. Wisdom. When I can't understand the aspects of the Lean Principles, I reach out to our Lean champion for help.

2. Strategy. I rarely find time to attend: Daily Huddles.

3. Mindfulness. I make sure I keep up with my performance by regularly checking my work; and

4. Leadership. When my team is confronted with a breakdown, I work hard with them to resolve the conflict. (FC)

The researchers linked the Product and Monitoring phases of the four-stage model of self-regulated learning by Winne and Hadwin (1998) to the performance dimension of the LLHA paths. The execution or performance and monitoring of a task are essential processes to achieve project goals (Winne and Perry, 2000). Therefore, it is essential to monitor the execution or performance of project strategies and develop an accurate measure of one's mastery. Project managers should regularly monitor the application and performance of strategies to develop an awareness of their efforts. Therefore, the research team developed questions to evaluate an individual's application and performance of Lean principles and techniques. The following are example questions from the assessment instrument:

1. Wisdom. When I can, I apply Lean principles in various aspects of the project.

2. Strategy. I try to find innovative design and processes to support the various aspects of the project.

3. Mindfulness. I make time in my agenda for retrospection of my own work; and

4. Leadership. I help my team members in implementing Lean principles as often as I can.

To identify areas of growth in individuals, they must have a metacognitive awareness of their skills and knowledge. Therefore, the research team included an awareness dimension in the LLHA to support individuals in evaluating their own knowledge and strategies for the development of metacognitive awareness. The dimension of awareness was aligned to the ‘cognitive evaluations’ phase of the four-stage model of self-regulated learning by Winne and Hadwin (1998). Furthermore, when developing this dimension, the researchers also aligned the awareness dimension to the Pintrich's (1991) scale for ‘Metacognitive Self-Regulation’. Developing an awareness of one's own strategies and knowledge can support an individual in controlling and regulating their behavior while learning about Lean (Pintrich, 1991). Tracking and monitoring learning promotes metacognitive control to find areas of improvement and assess the level of performance (Winne and Perry, 2000). By developing this awareness, one can improve their own performance in current and future learning efforts (Pintrich, 1991). This dimension is the most important, since the lack of awareness of one's own motivation, knowledge, effort and performance will lead an individual to have overconfidence in Lean principles and not pursuit continuous improvement. Furthermore, developing awareness of one's Lean journey can support individuals in avoiding the Dunning-Kruger effect (Kruger and Dunning, 1999), where novice employees tend to have higher confidence levels than experts, while having less skills and knowledge of a task. The research team came up with questions to evaluate an individual's awareness level. Some example questions from the assessment instrument:

1. Wisdom. Even though Lean principles are new to me, I believe I am doing well in the learning path.

2. Strategy. When I notice that I don't apply Lean techniques, I make sure I apply them the next chance I get.

3. Mindfulness. The monitoring of one's own journey is essential to becoming a master, so I perform knowledge assessments; and

4. Leadership. I don't really contribute to the monitoring of our work when we are: Identifying different types of waste.

Wisdom can be defined as the understanding, knowledge or experience (Oxford Dictionary, 2017) of a subject. With the LLHA, an individual can evaluate her/his knowledge or wisdom confidence level of Lean principles. Also, one can develop an awareness of their knowledge gaps. By developing knowledge awareness, one will be able to know which Lean principles to apply within the project. The dimensions in the Wisdom Path are knowledge, confidence in the various Lean principles; motivation and interest in developing Lean knowledge; effort regulation in understanding the principles; applying and performing them frequently; and an awareness of one's knowledge. The following are some of the Lean principles necessary for an IPM in projects, as defined by Seed (2015), that are being assessed in LLHA:

1. Value – An organization should focus on the overall project goals and outcomes for the customer rather than individual efforts to maximize value and quality of processes.

2. Waste – Stakeholders should identify and eliminate waste or non-value adding activities to optimize the output, reducing the additional cost/resources caused by it;

3. Continuous Improvement (Kaizen) – A Lean organization should strive for perfection through reflection and learning.

4. Respect for people – Lean demands respecting the input of each participant in project delivery as it helps reducing “Underutilized Employee Talent” type of waste and drive continuous improvement; and

5. Facilitation–Having an effective facilitator who promotes individual's engagement and collaborative learning in group meetings, which in turn leads to success through effective communication flow.

Lean Construction offers individuals and teams a variety of strategies and techniques to improve the process of delivering projects, by maximizing value and minimizing waste. Project managers must apply these strategies and techniques frequently to achieve proficiency and mastery. Evaluation of knowledge, motivation, efforts and efforts on the implementation of the strategies can add value to the processes and upstream the outcome of the tasks. The dimensions in the Strategy Path are knowledge confidence in the various Lean techniques; motivation and interest in applying Lean techniques; effort regulation in applying the strategies; applying and performing them frequently; and an awareness of one's journey in applying the strategies.

The LLHA assesses many Lean techniques, as defined by the Lean construction institute (Messner et al., 2019), such as:

1. Last Planner System® – A management system that measures the overall performance of a project through commitment to schedule tasks by the participants performing the work, facilitating collaborative project tracking and team involvement from the initial phase.

2. A3 Thinking – A collaborative decision analysis method that documents the problem and potential solutions on an A3-sized paper to visually display decisions. It supports the plan-do-check-adjust decision-making method.

3. 5 Whys – A process to determine the root cause of a problem using cause and effect relationship; and

4. Value Stream Mapping – A process to evaluate a sequence of activities to identify waste and value-adding steps.

Continuous monitoring activities support the identification of one's strengths and weaknesses to drive improvement in learning. One must monitor their Lean journey to achieve improvements. Such monitoring can happen during or at the end of learning or project delivery. Therefore, project managers must evaluate their learning and Lean journey by applying several monitoring strategies. With the Mindfulness Path, the researchers aimed at evaluating the following dimensions: knowledge confidence in various retrospective and monitoring techniques; motivation and interest in applying such techniques; effort regulation in monitoring the Lean journey; applying and performing the strategies; and an awareness of one's monitoring. The evaluated techniques, as defined by Seed (2015), for the Mindfulness path include:

1. Plus/Delta – A technique that aims at improving the group meetings or discussions to refine overall project processes. It helps determine the activities that bring value (Plus) and those which need improvement (Delta).

2. Retrospection – Retrospective activities review the completed actions and assess the output with the expected results. Regular monitoring aids in continuous improvement and value-adding tasks; and

3. Lessons Learned – Keeping a record of lessons learned is essential to determine the areas of improvement and success stories.

In addition to the goal of implementing and striving for mastery of Lean principles and techniques, one must also lead and support their team in achieving such a goal. Achieving such leadership status requires rigorous effort, motivation and strong foundational knowledge. A leader must be aware of the team members' strengths and weaknesses to guide them in achieving mastery. In the LLHA, the researchers provide several questions that evaluate one's leadership qualities and engagement with the team. Furthermore, the assessment evaluates one's drive in directing the learning and training efforts of the team. Several questions also evaluate an individual's efforts to create a safe environment based on trust and collaboration. In the Leadership path, the researchers evaluate the following dimensions: knowledge confidence in teaching and training others; motivation and interest in leading learning and management efforts; effort regulation in leading others; applying and performing leadership strategies; and an awareness of one's leadership style.

The combination of the instrument dimensions and path provides a comprehensive representation of the interaction of Lean with IPD concepts. To give an accurate measure of the dimensions for each path, the instrument included questions translating into 149 variables on a 5-point Likert scale from “Very true of me” to “Not at all true of me”. These variables were used in the instrument questionnaire to collect data and later analyzed based on the PCA.

The research team aligned questions within the different paths to questions for the different scales of the MSLQ to generate the LLHA instrument as captured in Table 2. For example, questions from the motivation, knowledge, efforts, performance and awareness dimensions are related to questions from self-efficacy, effort regulation, goal orientation and metacognition scales of the MSLQ questionnaire.

Like the self-regulated theories alignment, the authors aligned the LLHA questions to Lean principles and methods, as summarized in Table 3. This step was performed to strengthen the instruments' content validity. The Lean practices and techniques were extracted from Transforming Design and Construction by Seed (2015), which elaborates on the transformational change that the design and construction industry must undergo to deliver projects in an integrated and Lean environment. By validating the instrument's face and content validity, the research team proved that LLHA can be implemented in construction projects and teams.

The implementation of a new assessment instrument requires validation before it can be deployed industry-wide. Face validity is achieved through evaluation of the assessment tool by field experts (Moring, 2014). During this evaluation step, experts verify and test if the constructs being represented are valid and in accordance with practice (Nevo, 1985). The research team went through a series of steps to validate the instrument, as summarized in Figure 3. First, the research team validated LLHA by analyzing its face and content with five experts bearing over 50 years of combined experience in Lean Construction.

Additionally, the experts' feedback supported the team in excluding questions that resulted in tester's fatigue as recommended by Allen et al. (2023). To tackle content validity, as defined by Gall et al. (1996), the researchers performed a systematic alignment of the instrument with already validated content and questionnaires, which was highlighted in the previous section. By achieving content validity, the researchers provide an accurate measure of the intended content and theory (Gall et al., 1996). Therefore, the authors validated that the questions included in self-regulation theories and Lean principles.

Based on the feedback received from the experts, the LLHA was verified using three IPD projects located within the US construction industry. The first project was a hotel resort in its 90% design development phase completed. The project team was composed of the owner's representatives, architect, designers and general contractor bound by a multiparty Integrated Form of Agreement (IFOA). The second project was an entertainment facility at 50% construction phase. The project team was composed of an integrated team where the owner's representatives, architects, designers, general contractors and specialty trade contractors also signed an Integrated Form of Agreement (IFOA). The third project was also an entertainment facility, with its 90% construction phase completed. The project team was composed of an integrated team with the owner's representatives, architects, designers and general contractor as part of the IFOA.

The participants from the main IFOA signatories that included the owner's representatives, design architects, engineers and general contractors or construction managers were given the instrument to use for the assessment. The instrument was also used to collect data from other project participants who were not part of the IFOA but participated in the integrated design and construction process of the projects, such as specialist trade partners and consultants, commissioners and specialized vendors. From these projects, the researchers were able to collect a total of n = 123 responses. Figure 4 provides a summary of the distribution of responses received from the participants involved in the study. The previous IPD experience of participants was not collected as part of this study. Based on the data collected from the three case study project participants, a PCA was performed. A high number of owner's participation was due to high response from the owner's representatives (see Figure 5).

A PCA was performed for the dataset collected using the LLHA instrument from the n = 123 participants. The IBM SPSS Statistics Version 29.0.0.0 (241) was used for the PCA. A dimension reduction was conducted using factor analysis to reduce the large set of variables to a few influential components. The dataset, consisting of 149 variables from a sample size of 123 participants (n = 123), was analyzed using PCA as shown in Figure 1. The dimension reduction process narrowed down the factors to the most influential under each of the categories. For each of the five (LLHA) categories; wisdom, strategy, mindfulness, leadership and integration, the 149 factors, classified under either knowledge, effort, performance, awareness and motivation, were reduced to only the most influential components.

The first round of analysis generated a maximum of eight (8) components accounting for up to 88% of the variance in the output. A fixed number of components was used for the second round of factor analysis, to generate five (5) components under each category, as summarized in Figure 5. The analysis sought a correlation between the assessment dimensions: motivation, knowledge, effort, performance and awareness, and the five assessment paths: wisdom, strategy, mindfulness, leadership and integration. Under each category, the correlation coefficients from the factor analysis were used to identify the variables with the highest correlation. The ranking of these influential components based on the rotated matrix coefficients generated influential aspects that project managers should consider when assessing their Lean learning journey. The initial analysis included 149 variables based on the questions included in the assessment questionnaire. These variables in the study were reduced using the factor analysis reduction technique.

Based on the analysis, the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy for the variables under each category ranges between 0.8 and 0.9, indicating meritorious according to Kaiser's 1974 classification (Kaiser, 1974; Kaiser and Rice, 1974). This signifies that the samples are adequate to run the principal component analysis. The Bartlett's test of sphericity is also <0.001 for all the analysis categories as summarized in Table 4. The null hypothesis is therefore rejected for the five categories, thus not all the variables are correlated, supporting the suitability of the PCA for the dataset.

The extracted components were then ranked to determine the most influential component under each category. This ranking was based on the rotated component matrix scores for each of the five components within the five categories of wisdom, strategy, mindfulness, leadership and integration. The outcome of the rotated component matrix scores indicates a high correlation between the components and the categories, exceeding 0.5. These outcomes have been summarized in Table 5. For each of the categories, the highest correlations were identified.

The components score under each category was then linked to the specific variable as derived from the rotated matrix output tables in SPSS. According to the output, the ability of an individual to apply Lean principles, having knowledge on look-ahead planning, tendency to perform retrospective reviews, supporting collaborative teams based on respect and their ability to foster trust within the project team are found to be the most influential variables in the Lean journey of individuals based on the LLHA instrument. However, awareness of successful methods of collaboration and sharing of ideas among integrated teams is noticeably lower, representing the lowest correlation from the analysis. Table 6 summarizes these factors under each category. The highlighted components represent the most influential variables under each category.

Among the components presented in Table 3, leadership and motivation that cultivates team members' ability to develop a collaborative team that is founded on mutual respect have the highest correlation based on the LLHA instrument, with a score of 0.951. This demonstrates the significance of team motivation and leadership when providing resources to support team members' Lean journey and their self-regulated learning.

The performance of the participants based on the correlation of the five influential variables under each category was compared to the participants' areas of specialization. Although project owners, architects, engineers, general and trade contractors have the same project goals, they also have different areas of emphasis when applying Lean principles within the integrated project delivery setting. Owners tend to prioritize the existence of a collaborative team based on respect and trust. Designers and contractors, however, prioritize trust within the team, while trade partners tend to focus on collaboration due to the interdependence of the design components. Figure 6 summarizes these differences in areas of focus.

Lean design and construction principles were introduced to the construction industry more than two decades ago, yet project management practices are still anchored on the traditional siloed methods. According to audit reports, the project-based and fragmented delivery leads to poor performance in the construction industry (McKinsey, 2020). To eliminate the poor performance, project teams need to understand the core concepts of integration in the delivery of construction projects (Franz and Roberts, 2022), a key requirement when implementing Lean in relational IPD projects. IPD contracts mainly focus on aligning the interests of participants through shared risks, rewards and collaborative incentives. Lean methods, tools and techniques provide a mechanism to make the collaboration process consistent (Herrera et al., 2021).

Understanding of Lean by the project participants in charge of managing different aspects of these IPD projects is therefore crucial (Laurent and Leicht, 2019). This understanding of Lean can be enhanced through self-regulated learning. However, there are currently limited tools specific to assessing progress in self-regulated Lean learning journey in IPD projects. This study sought to address this by developing an instrument that allows practitioners to assess their level of understanding of Lean as applied in integrated construction projects. The development of the LLHA instrument was based on an existing self-regulated learning and knowledge assessment framework. The Winne and Hadwin (1998) four-stage model of self-regulated learning and Pintrich et al. (1993) Motivated Strategies for Learning Questionnaire (MSLQ) were leveraged to generate the dimensions and components of the LLHA instrument.

Awareness of the level of understanding of Lean plays a significant role in project participants' ability to advance their Lean learning and subsequent application in projects. The LLHA instrument gauges the level of motivation, knowledge, effort, performance and awareness of Lean concepts among IPD practitioners. In self-regulated learning, motivation enables the learner to initiate the process of developing understanding for the concept, in this case, the Lean principles, methods and tools. Based on this understanding, the learner acquires knowledge to perform Lean actions and evaluate their performance to identify areas where more effort is required. Through a series of questions, the instrument assesses the areas of strength of the participants in four paths: wisdom when implementing the principle, strategies used to advance their learning process, mindfulness of personal progress in the learning journey and team leadership and integration based on the Lean concepts learned. The results of the assessment give the learner an indication of the Lean concepts they have a better understanding of and the areas where effort may still need to be added for improvement.

The LLHA assessment tool was evaluated for face and content validity through interviews with five experts with over 50 years of cumulative experience with Lean construction. To validate the instrument, data were collected from participants in three IPD projects in the USA. A total of n = 123 responses were received from participants of these three projects. Principal components were extracted through a factor analysis of the instrument components. The analysis demonstrates that the knowledge key areas that participants require mastery of to successfully implement Lean in IPD projects include the ability to collaborate with other participants, the ability to foster an environment based on trust, confidence in applying the principles of Lean and the ability to reflect on their performance in implementing Lean in the IPD projects they are part of. Understanding of Lean methods and tools, such as Look-ahead planning, value stream mapping, developing A3s, performing a root cause analysis and reflecting on lessons learned, is also crucial for Lean-IPD project participants.

The results also indicate that different participant groups implement different concepts of Lean with different levels of emphasis based on their roles and overall interests in the project. This may be explained by the difference in organizational goals of the companies involved, which often influence their interests in the project (Choi et al., 2019). The difference in interest may generate the disparity in their level of understanding of the Lean concepts and methods. The level of commitment to learning different aspects of Lean may be driven by the performance improvements desired by the individual organizations.

Based on the principal component analysis, owners tend to prioritize project team's awareness of collaboration within the team, based on respect and trust. Trade partners also prioritize collaboration within the team. This can be explained by the high degree of interdependency among work performed by different trades that require coordination and teamwork to achieve success. Such interdependencies also call for mutual respect and trust among the trades (Ferstad et al., 2023). On the other hand, Architects, designers and contractors have a better understanding of fostering a collaborative working environment that is founded on trust.

Although integrated project teams consist of participants from different disciplines with high specialization levels, they work towards the common goal of delivering the project within defined goals (Choi et al., 2019). However, when applying Lean principles, these participants have different levels of understanding of Lean, an observation that likely results from the individual- and company-level interests. The tool helps participants identify their individual performance in implementing Lean and the areas that require emphasis. Collective results from the team can also reveal the areas participants need to improve to obtain better results. A higher level of awareness on the areas of strength and low performance can then trigger the team to identify the training needed for the different Lean principles, methods and tools.This study also reveals that Lean-IPD project teams focus on creating a collaborative environment for project execution. However, there is a need to further emphasize on enhancing their understanding of Lean methods and tools and sharing information and knowledge beyond their areas of specialization.

The authors recognize the existence of three main factors that may limit the study. First, the data was collected from only three IPD projects, based only in the US construction industry. Further studies can be used to validate the instrument by collecting data from more IPD projects from different geographical contexts. The number of participants was also limited, with more responses from participants from the owners' side compared to other disciplines. This may not present an accurate representation of construction project participants involved in IPD projects. Further studies can include more representative samples in validating the instrument. Second, the study design did not consider the differences in current levels of practical understanding of IPD concepts among participants at the time of data collection. The instrument does not consider past experience with IPD projects but focuses on the understanding from their self-regulated efforts in learning Lean concepts and how they are applying these in the current IPD projects. Future research can modify these aspects of the instrument to capture the level of experience with Lean-IPD as part of the data collection and analysis process. The instrument was also validated with IPD projects and specifically those based in the US construction industry. Further studies can validate the use of this instrument with participants in non-IPD projects implementing Lean principles, methods and tools. The instrument can also be piloted and validated for application in geographical contexts outside the USA by analyzing the face and content validity with construction industry and Lean experts outside the USA and validating with projects located in other geographical markets. Lastly, a repeated test was also not conducted to gauge the changes in the learning journey after the assessment. Future studies may consider conducting the LLHA using a pre- and post-assessment survey to determine the impact of the assessment on learning progress among participants.

The continuous adoption of delivery methods in the construction industry demands current and future project participants to learn a new range of principles, techniques and skills. This shift requires continuous learning and evaluation to identify the knowledge gaps. Several instruments exist to support assessment of strengths and weakness, personality type and attitude but are not specific to the new delivery systems in construction projects, specifically IPD projects that integrate Lean philosophies. These assessment instruments cannot, therefore, guide the awareness of team participants' knowledge gaps and ability to self-regulate their learning efforts.

This study presents a Lean Learning Health Assessment (LLHA) instrument designed to support an individual's awareness of their Lean journey and identify areas of improvement. The LLHA instrument is available on the Lean Construction Institute's official website, in the learning section's tools and technologies tab. The assessment instrument was validated through subject matter expert interviews and administering the questionnaire used in its development to n = 123 participants involved in three IPD projects in the USA. The researchers then conducted a principal component analysis to identify which Lean components included in the instrument significantly captured the assessment instrument's dimensions. The analysis demonstrates that leadership and collaboration based on respect and trust are the key factors of emphasis among participants in self-regulated Lean learning. On the other hand, complete integration and knowledge exchange remain an area with lower learning outcomes among participants in integrated teams.

This study contributes to the current body of knowledge by providing an instrument designed for the assessment of project teams level of awareness of Lean and to self-regulate their Lean learning journey. By testing the instrument using several integrated project participants, the study identifies potential areas of high awareness among such integrated project teams and the lacking areas. This instrument can support industry members in their learning efforts by guiding them in identifying the knowledge gaps to strengthen their Lean learning journey.

Practitioners can benefit from this instrument by deploying it in their current projects, implementing Lean by assessing learning efforts among project participants. This assessment can be achieved by distributing the instrument to participants and analyzing the level of variations in understanding of Lean among these participants, including the knowledge gaps across each of the Lean methods and tools evaluated by the instrument.

As part of future research, the authors will implement the assessment instrument in additional projects and assess the participants over time as they learn and implement Lean in IPD projects. Furthermore, the research team aims to expand this assessment instrument to further address integration factors within a team, such as trust, accountability and mutual goals. The team also aims to develop an automated curriculum generator for individuals who take the assessment. Such a curriculum generator would identify the areas where an individual has low confidence or performance and provide them with suggestions of resources necessary to tackle such gaps.

The authors would like to thank all the participants and project teams that made this research study possible.