Table I Realist systematic review... | Emerald Publishing

Table I

Realist systematic review criteria

Step 1: clarify scope criteria	Researcher’s response
a1. Identify the research question	1. How are multiteam systems structured? 2. What is the best intervention for organizing multiteam systems? 3. What is the best leadership model for multiteam systems?
a2. Nature and content of the intervention	1. Simple and complicated problems 2. Complex problems
a3. Circumstances or context for its use	1. What are the different interventions required for complicated and complex problems? Are there distinct differences? 2. The context is within the Toyota Connected (TC) system, utilizing teams trained in agile systems and operating agile teams in a real, open workplace, with proven successes
a4. Policy intentions or objectives	1. A new intervention for managing/organizing multiteam systems is desired and will be implemented and tested once a new intervention has been identified and agreed upon by the researchers and stakeholders
b1. Refine the purpose of the review	1. Evaluate program to identify problems and bottlenecks 2. Identify other comparable interventions that may work better 3. Identify all stakeholders involved
b2. Theory integrity – does the intervention work as predicted?	1. Test new intervention to identify if it works better than original
b3. Theory adjudication – which theories fit best?	1. Compare interventions, or continue modifying interventions, until a “best intervention” is identified
b4. Comparison – how does the intervention work in different settings, for different groups?	1. Test the new intervention using different samples and settings
b5. Reality testing – how does the policy intent of the intervention translate into practice?	1. Evaluate the testing of the new intervention
c1. Articulate key theories to be explored	1. Identify theory, or theories, that will be utilized to structure the chosen intervention(s) 2. Theories that address the multiteam level of analysis will be collected and mapped 3. Theories associated to the traditional, multilevel, or other levels of analysis (individual, team and organization) will be collected and mapped
c2. Draw up a “long list” of relevant program theories by exploratory searching (see Step 2)	1. Identification of a long list of theories that explain/describe each step in the intervention’s process will be presented from the literature
c3. Group, categorize or synthesize theories	1. The theories presented in c2 will be categorized and synthesized based on their key characteristics and advantages/disadvantages and mapped according to the intervention stage that they represent
c4. Design a theoretically based evaluative framework to be “populated” with evidence	1. Design evaluation mechanisms to measure each of the identified theories 2. Design a testing strategy to measure the appropriate theory during each stage of the intervention. Know when to test and what to measure

Step 1: clarify scope criteria	Researcher’s response
a1. Identify the research question	1. How are multiteam systems structured?2. What is the best intervention for organizing multiteam systems?3. What is the best leadership model for multiteam systems?
a2. Nature and content of the intervention	1. Simple and complicated problems2. Complex problems
a3. Circumstances or context for its use	1. What are the different interventions required for complicated and complex problems? Are there distinct differences?2. The context is within the Toyota Connected (TC) system, utilizing teams trained in agile systems and operating agile teams in a real, open workplace, with proven successes
a4. Policy intentions or objectives	1. A new intervention for managing/organizing multiteam systems is desired and will be implemented and tested once a new intervention has been identified and agreed upon by the researchers and stakeholders
b1. Refine the purpose of the review	1. Evaluate program to identify problems and bottlenecks2. Identify other comparable interventions that may work better3. Identify all stakeholders involved
b2. Theory integrity – does the intervention work as predicted?	1. Test new intervention to identify if it works better than original
b3. Theory adjudication – which theories fit best?	1. Compare interventions, or continue modifying interventions, until a “best intervention” is identified
b4. Comparison – how does the intervention work in different settings, for different groups?	1. Test the new intervention using different samples and settings
b5. Reality testing – how does the policy intent of the intervention translate into practice?	1. Evaluate the testing of the new intervention
c1. Articulate key theories to be explored	1. Identify theory, or theories, that will be utilized to structure the chosen intervention(s)2. Theories that address the multiteam level of analysis will be collected and mapped3. Theories associated to the traditional, multilevel, or other levels of analysis (individual, team and organization) will be collected and mapped
c2. Draw up a “long list” of relevant program theories by exploratory searching (see Step 2)	1. Identification of a long list of theories that explain/describe each step in the intervention’s process will be presented from the literature
c3. Group, categorize or synthesize theories	1. The theories presented in c2 will be categorized and synthesized based on their key characteristics and advantages/disadvantages and mapped according to the intervention stage that they represent
c4. Design a theoretically based evaluative framework to be “populated” with evidence	1. Design evaluation mechanisms to measure each of the identified theories2. Design a testing strategy to measure the appropriate theory during each stage of the intervention. Know when to test and what to measure

Source: Key steps in realist review from Pawson et al. (2005)