List of case tests along with their corresponding validity and reliability instruments
| Test case | Instrument/action | Instrument/action details | Code |
|---|---|---|---|
| Content validity | Ask experts for support | …to revise the questions in the interview script | CV1A |
| …on methodological issues | CV1B | ||
| …when attending conferences | CV1C | ||
| …to review the work’s literature | CV1D | ||
| …to evaluate the study’s comparable results | CV1E | ||
| …to review the study’s findings | CV1F | ||
| …academia and the market | CV1G | ||
| …to define the case selection criteria | CV1H | ||
| …to obtain technical information in the context of analysis | CV1I | ||
| …during the exploratory phase of the study | CV1J | ||
| …to evaluate emerging concepts and themes | CV1J | ||
| Carry out a pilot test | …to refine the interview script | CV1L | |
| …to refine the questionnaire | CV1M | ||
| …to check the integrity of the framework | CV1N | ||
| Send a copy of the interview script to the interviewees in advance | – | CV2 | |
| Create a diagram or illustration of the structure to help explain it during data collection | – | CV3 | |
| Perform face validity | – | CV4 | |
| Adjust/improve the interview script during the data collection process | – | CV5 | |
| Research previous works in the literature related to the topic (they support the literature) | – | CV6 | |
| Conduct postsurvey discussion | – | CV7 | |
| In focus groups, ensure everyone’s participation | – | CV8 | |
| In focus groups, verbal and nonverbal attention shifts from dominant participants | – | CV9 | |
| Prepare the script in such a way as to avoid unnecessary ambiguity | – | CV10 | |
| Prepare the script in such a way as to avoid discrediting any interviewee | – | CV11 | |
| Use the researcher’s independence and neutrality to create an informal interview situation | – | CV12 | |
| Construct validity | Perform triangulation | Data | COV1A |
| Researchers | COV1B | ||
| Methodological | COV1C | ||
| Theoretical | COV1D | ||
| Include multiple sources of evidence | – | COV2 | |
| Have key informants review the draft | …the study participants themselves | COV3A | |
| …being experts | COV3B | ||
| – | |||
| Establish a chain of evidence | – | COV4 | |
| Identify units of analysis and acquire data to measure them | – | COV5 | |
| Elaborate the conceptual framework based on the literature and the experience of the researchers | – | COV6 | |
| Perform factorial analysis | – | COV7 | |
| Define the concept, case and context under study | – | COV8 | |
| Ensure congruence between research question and method | – | COV9 | |
| Use theory to develop constructs | – | COV10 | |
| Define your theoretical concepts explicitly from the initial design of the research and “maintain a uniform definition throughout the research process” | – | COV11 | |
| Draw up a well-written interview script | – | COV12 | |
| The interviewer has previous experience in conducting surveys | – | COV13 | |
| Use an expert interview | – | COV14 | |
| Check the transcript of the interviews and compare them with the recordings | – | COV15 | |
| External validity | Seeking analytical generalization | – | EV1 |
| Using multiple cases | – | EV2 | |
| Use replication logic | Literal | EV3A | |
| Theoretical | EV3B | ||
| Use theory in single case studies | – | EV4 | |
| Perform analysis between cases | – | EV5 | |
| Select more than one group of participants | …to ensure a cross-section of opinions | EV6A | |
| …and collect data over different periods of time | EV6B | ||
| …to limit elite bias | EV6C | ||
| – | |||
| Carry out theoretical sampling | – | EV7 | |
| Internal validity | Perform pattern matching | – | IV1 |
| Interview only key informants | – | IV2 | |
| Building causal evidence | – | IV3 | |
| Building explanations | – | IV4 | |
| Examining alternative propositions (rival explanation) | – | IV5 | |
| Purposely select the sample | – | IV6 | |
| Building logical models | – | IV7 | |
| Matching theory and field | – | IV8 | |
| Peer debriefing | – | IV9 | |
| Carry out action research as part of a case study | – | IV10 | |
| Search for patterns | – | IV11 | |
| Create diagrams to ensure internal consistency of the data collected | – | IV12 | |
| Carry out inter-rater reliability testing | – | IV13 | |
| Asking respondents to explain their answers to the Likert scale questions | – | IV14 | |
| Perform hierarchical interpretation and explanation of the inductive logical relationship by the research framework | – | IV15 | |
| Record alternative explanations | – | IV16 | |
| Record the sampling criteria in the study protocol | – | IV17 | |
| Use cognitive mapping to convey causal connections | – | IV18 | |
| Use regression modeling to converge sources of evidence | – | IV19 | |
| Conduct discriminant validity | – | IV20 | |
| Perform member checking | – | IV21 | |
| Reliability | Record the interviews | – | RE1 |
| Create a structured research protocol | – | RE2 | |
| Carry out an iterative process between data, concepts and coding | – | RE3 | |
| Transcribe all interviews verbatim | – | RE4 | |
| Create a case study database | – | RE5 | |
| Avoid sectarian views (interpretation bias) | – | RE6 | |
| Create a field diary during data collection (field notes) | – | RE7 | |
| Carry out independent coding of the data | – | RE8 | |
| Recode or combine categories | – | RE9 | |
| Provide a detailed description of the data collection and analysis processes | – | RE10 | |
| Look for significant parallels in the findings of the multiple data sources | – | RE11 | |
| Confirm the coding of the data among the authors (coding process) | – | RE12 | |
| Create an interview script with open-ended questions | – | RE13 | |
| Carry out a detailed description of the research context | – | RE14 | |
| Carry out in-depth data collection | – | RE15 | |
| Conduct an audit trail to record changes and limitations | – | RE16 | |
| Return to the field more than once. Collect data more than once (for nonlongitudinal studies) | – | RE17 | |
| Conduct a systematic interpretation of the data or field notes | – | RE18 | |
| Perform intercoder agreement (member check) | – | RE19 | |
| Make notes for cases where recording was not authorized | – | RE20 | |
| Perform intercoder reliability | – | RE21 | |
| Keep the results consistent with reality | – | RE22 | |
| Perform Cronbach’s alpha test | – | RE23 | |
| Carry out the data collection process under similar circumstances for all interviews | – | RE24 | |
| Carry out complementary questioning to obtain a complete view of a given subject | – | RE25 | |
| Make a conscious effort to keep the researcher’s interpretations separate from the coding process | – | RE26 | |
| Conduct an automated content analysis, seeking to remove subjective interpretations | – | RE27 | |
| Perform composite reliability | – | RE28 | |
| Perform indicator reliability | – | RE29 | |
| Make the data available to external reviewers | – | RE30 | |
| Explain the intention of the study to the interviewee in advance | – | RE31 | |
| Clearly state the position of the researchers about the situation under investigation | – | RE32 | |
| Make the author of the study available to provide additional information on the selected cases | – | RE33 | |
| Omit variables that do not meet the theoretical criteria | – | RE34 | |
| Organize both primary and secondary data in an Excel spreadsheet | – | RE35 | |
| Standardize the description of the cases | – | RE36 | |
| Standardize field notes | – | RE37 | |
| Carry out a “confirmability audit” | – | RE38 | |
| Carry out a systematic thematic analysis | – | RE39 |
| Test case | Instrument/action | Instrument/action details | Code |
|---|---|---|---|
| Content validity | Ask experts for support | …to revise the questions in the interview script | CV1A |
| …on methodological issues | CV1B | ||
| …when attending conferences | CV1C | ||
| …to review the work’s literature | CV1D | ||
| …to evaluate the study’s comparable results | CV1E | ||
| …to review the study’s findings | CV1F | ||
| …academia and the market | CV1G | ||
| …to define the case selection criteria | CV1H | ||
| …to obtain technical information in the context of analysis | CV1I | ||
| …during the exploratory phase of the study | CV1J | ||
| …to evaluate emerging concepts and themes | CV1J | ||
| Carry out a pilot test | …to refine the interview script | CV1L | |
| …to refine the questionnaire | CV1M | ||
| …to check the integrity of the framework | CV1N | ||
| Send a copy of the interview script to the interviewees in advance | – | CV2 | |
| Create a diagram or illustration of the structure to help explain it during data collection | – | CV3 | |
| Perform face validity | – | CV4 | |
| Adjust/improve the interview script during the data collection process | – | CV5 | |
| Research previous works in the literature related to the topic (they support the literature) | – | CV6 | |
| Conduct postsurvey discussion | – | CV7 | |
| In focus groups, ensure everyone’s participation | – | CV8 | |
| In focus groups, verbal and nonverbal attention shifts from dominant participants | – | CV9 | |
| Prepare the script in such a way as to avoid unnecessary ambiguity | – | CV10 | |
| Prepare the script in such a way as to avoid discrediting any interviewee | – | CV11 | |
| Use the researcher’s independence and neutrality to create an informal interview situation | – | CV12 | |
| Construct validity | Perform triangulation | Data | COV1A |
| Researchers | COV1B | ||
| Methodological | COV1C | ||
| Theoretical | COV1D | ||
| Include multiple sources of evidence | – | COV2 | |
| Have key informants review the draft | …the study participants themselves | COV3A | |
| …being experts | COV3B | ||
| – | |||
| Establish a chain of evidence | – | COV4 | |
| Identify units of analysis and acquire data to measure them | – | COV5 | |
| Elaborate the conceptual framework based on the literature and the experience of the researchers | – | COV6 | |
| Perform factorial analysis | – | COV7 | |
| Define the concept, case and context under study | – | COV8 | |
| Ensure congruence between research question and method | – | COV9 | |
| Use theory to develop constructs | – | COV10 | |
| Define your theoretical concepts explicitly from the initial design of the research and “maintain a uniform definition throughout the research process” | – | COV11 | |
| Draw up a well-written interview script | – | COV12 | |
| The interviewer has previous experience in conducting surveys | – | COV13 | |
| Use an expert interview | – | COV14 | |
| Check the transcript of the interviews and compare them with the recordings | – | COV15 | |
| External validity | Seeking analytical generalization | – | EV1 |
| Using multiple cases | – | EV2 | |
| Use replication logic | Literal | EV3A | |
| Theoretical | EV3B | ||
| Use theory in single case studies | – | EV4 | |
| Perform analysis between cases | – | EV5 | |
| Select more than one group of participants | …to ensure a cross-section of opinions | EV6A | |
| …and collect data over different periods of time | EV6B | ||
| …to limit elite bias | EV6C | ||
| – | |||
| Carry out theoretical sampling | – | EV7 | |
| Internal validity | Perform pattern matching | – | IV1 |
| Interview only key informants | – | IV2 | |
| Building causal evidence | – | IV3 | |
| Building explanations | – | IV4 | |
| Examining alternative propositions (rival explanation) | – | IV5 | |
| Purposely select the sample | – | IV6 | |
| Building logical models | – | IV7 | |
| Matching theory and field | – | IV8 | |
| Peer debriefing | – | IV9 | |
| Carry out action research as part of a case study | – | IV10 | |
| Search for patterns | – | IV11 | |
| Create diagrams to ensure internal consistency of the data collected | – | IV12 | |
| Carry out inter-rater reliability testing | – | IV13 | |
| Asking respondents to explain their answers to the Likert scale questions | – | IV14 | |
| Perform hierarchical interpretation and explanation of the inductive logical relationship by the research framework | – | IV15 | |
| Record alternative explanations | – | IV16 | |
| Record the sampling criteria in the study protocol | – | IV17 | |
| Use cognitive mapping to convey causal connections | – | IV18 | |
| Use regression modeling to converge sources of evidence | – | IV19 | |
| Conduct discriminant validity | – | IV20 | |
| Perform member checking | – | IV21 | |
| Reliability | Record the interviews | – | RE1 |
| Create a structured research protocol | – | RE2 | |
| Carry out an iterative process between data, concepts and coding | – | RE3 | |
| Transcribe all interviews verbatim | – | RE4 | |
| Create a case study database | – | RE5 | |
| Avoid sectarian views (interpretation bias) | – | RE6 | |
| Create a field diary during data collection (field notes) | – | RE7 | |
| Carry out independent coding of the data | – | RE8 | |
| Recode or combine categories | – | RE9 | |
| Provide a detailed description of the data collection and analysis processes | – | RE10 | |
| Look for significant parallels in the findings of the multiple data sources | – | RE11 | |
| Confirm the coding of the data among the authors (coding process) | – | RE12 | |
| Create an interview script with open-ended questions | – | RE13 | |
| Carry out a detailed description of the research context | – | RE14 | |
| Carry out in-depth data collection | – | RE15 | |
| Conduct an audit trail to record changes and limitations | – | RE16 | |
| Return to the field more than once. Collect data more than once (for nonlongitudinal studies) | – | RE17 | |
| Conduct a systematic interpretation of the data or field notes | – | RE18 | |
| Perform intercoder agreement (member check) | – | RE19 | |
| Make notes for cases where recording was not authorized | – | RE20 | |
| Perform intercoder reliability | – | RE21 | |
| Keep the results consistent with reality | – | RE22 | |
| Perform Cronbach’s alpha test | – | RE23 | |
| Carry out the data collection process under similar circumstances for all interviews | – | RE24 | |
| Carry out complementary questioning to obtain a complete view of a given subject | – | RE25 | |
| Make a conscious effort to keep the researcher’s interpretations separate from the coding process | – | RE26 | |
| Conduct an automated content analysis, seeking to remove subjective interpretations | – | RE27 | |
| Perform composite reliability | – | RE28 | |
| Perform indicator reliability | – | RE29 | |
| Make the data available to external reviewers | – | RE30 | |
| Explain the intention of the study to the interviewee in advance | – | RE31 | |
| Clearly state the position of the researchers about the situation under investigation | – | RE32 | |
| Make the author of the study available to provide additional information on the selected cases | – | RE33 | |
| Omit variables that do not meet the theoretical criteria | – | RE34 | |
| Organize both primary and secondary data in an Excel spreadsheet | – | RE35 | |
| Standardize the description of the cases | – | RE36 | |
| Standardize field notes | – | RE37 | |
| Carry out a “confirmability audit” | – | RE38 | |
| Carry out a systematic thematic analysis | – | RE39 |
As a benefit of your subscription, you can share temporary access to restricted articles.
Each link will stop working after 30 days or 10 uses. You may create up to 10 links in a 30 day period.
Please sign in to your personal account to gift article access.
As a benefit of your subscription, you can share temporary access to restricted articles.
Each link will stop working after 30 days or 10 uses. You may create up to 10 links in a 30 day period.
Gift articles remaining: --
Each link will stop working after 30 days or 10 uses. You may create up to 10 links in a 30 day period.
Gift articles remaining: --
As a benefit of your subscription, you can share temporary access to restricted articles.
Each link will stop working after 30 days or 10 uses.
You have reached the limit of 10 links within a 30 day period.
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.