Table A5. Research outcomes assessment... | Emerald Publishing

Table A5.

Research outcomes assessment (ask only if they are aware of the project) (purpose to determine extent of outcome realization and research influence on knowledge or social process contributions around [topic])

Main question	Probes	Intent: what we are trying to find out Do NOT ask these directly
15. What contributions do you think [the researcher]’s project has made to young people’s engagement in post-conflict truth telling/transitional justice processes?	• Changes in knowledge/understanding? • Changes in attitudes? • Changes in skills? • Changes in relationships? • Changes in behaviour? • At what level do these changes mostly occur? (i.e. organizational, individual, governmental, policy, practice) • When did these changes occur? (during, post-project) • What are the implications of these changes? • Were there any negative outcomes of this project? If yes, please describe. • Probe for specific outcomes the researcher thought the informant could speak to. • What do you think the researcher did well to realize these results? • How accessible did you find the results and communication during the process? • Do you think the research can be transferred to other contexts?	*Understanding the respondent’s opinion about the contributions of the research.* Finding out the respondent’s opinion on the student’s research contributions (without leading to specific outcomes). Can give an indication of the utility of the research. Finding out how the student’s research is/was perceived and conceptualized by interviewees to get an overall characterization of the change process. This will help us construct narratives about alternative and/or supplementary theories of change. Finding out about the explicit outcomes/impacts of the project in question anywhere (in the world) of which the informant is aware, not just within their own work/organization. QAF: Rel6, Rel7, Cre7, Cre8, Cre10, Leg3, Eff1, Eff2, Eff3, Eff4
16. Has the research contributed to or influenced your work on the topic?	• What were the most important things you learned? • Have there been any positive or negative impacts on knowledge, awareness, policy, capacity, or practice? • In what ways? [ask for examples] • [If respondent mentions knowledge, ask about what knowledge product it came from]	*Understanding how the student’s research has influenced their work (re: the topic of focus).* Finding out about linkages between project and informant’s work on the topic of focus*, and whether the research has contributed to changes in policy and practice, the debate, awareness in the topic, knowledge, capacity, or any other type of contributions. Getting a sense whether the change is perceived as positive or negative. QAF: Rel5, Eff1, Eff2, Eff3, Eff4
17. If there was more time and resources available, what do you think [the researcher] could have done differently to produce more useful findings and/or change?	• Why do you think these would be useful? [ask for examples] • How do you think [the researcher] could have integrated these into their project? • Why do you think this [suggestion] was not done? • Do you think resources were efficiently and appropriately allocated?	*Understanding alternative ToCs and perspectives of the research potential beyond what it did realize/intended to, and other opportunities.* Hold to the end of the interview – if the interviewee starts talking about it at the beginning, please lead them back to any of the questions above and ask to return to the question. This Q allows participants to give feedback to the project and helps identify gaps/challenges, but we know many of the problems already and do not want to let this dominate/ mislead the main focus of the interview. Use this opportunity to increase the depth of any previous answers by probing and relating this question to any other points informants raise – if/when appropriate. QAF: Rel3, Rel5, Rel5, Rel7, Cre1, Leg3
18. What would have happened in young people’s engagement in post-conflict truth telling/transitional justice processes if this research had not been conducted?	• Probe to clarify if needed (the role of the project in improving collaboration, social networks, participation, engagement, etc.)	*Testing “zero hypothesis”.* Using a different angle to understand the true influence of ICRAF by asking what would be different had ICRAF not done its work. QAF: Eff4

Main question	Probes	Intent: what we are trying to find out Do NOT ask these directly
15. What contributions do you think [the researcher]’s project has made to young people’s engagement in post-conflict truth telling/transitional justice processes?	• Changes in knowledge/understanding?• Changes in attitudes?• Changes in skills?• Changes in relationships?• Changes in behaviour?• At what level do these changes mostly occur? (i.e. organizational, individual, governmental, policy, practice)• When did these changes occur? (during, post-project)• What are the implications of these changes?• Were there any negative outcomes of this project? If yes, please describe.• Probe for specific outcomes the researcher thought the informant could speak to.• What do you think the researcher did well to realize these results?• How accessible did you find the results and communication during the process?• Do you think the research can be transferred to other contexts?	Understanding the respondent’s opinion about the contributions of the research.Finding out the respondent’s opinion on the student’s research contributions (without leading to specific outcomes). Can give an indication of the utility of the research.Finding out how the student’s research is/was perceived and conceptualized by interviewees to get an overall characterization of the change process. This will help us construct narratives about alternative and/or supplementary theories of change.Finding out about the explicit outcomes/impacts of the project in question anywhere (in the world) of which the informant is aware, not just within their own work/organization.QAF: Rel6, Rel7, Cre7, Cre8, Cre10, Leg3, Eff1, Eff2, Eff3, Eff4
16. Has the research contributed to or influenced your work on the topic?	• What were the most important things you learned?• Have there been any positive or negative impacts on knowledge, awareness, policy, capacity, or practice?• In what ways? [ask for examples]• [If respondent mentions knowledge, ask about what knowledge product it came from]	Understanding how the student’s research has influenced their work (re: the topic of focus).Finding out about linkages between project and informant’s work on the topic of focus*, and whether the research has contributed to changes in policy and practice, the debate, awareness in the topic, knowledge, capacity, or any other type of contributions. Getting a sense whether the change is perceived as positive or negative.QAF: Rel5, Eff1, Eff2, Eff3, Eff4
17. If there was more time and resources available, what do you think [the researcher] could have done differently to produce more useful findings and/or change?	• Why do you think these would be useful? [ask for examples]• How do you think [the researcher] could have integrated these into their project?• Why do you think this [suggestion] was not done?• Do you think resources were efficiently and appropriately allocated?	Understanding alternative ToCs and perspectives of the research potential beyond what it did realize/intended to, and other opportunities.Hold to the end of the interview – if the interviewee starts talking about it at the beginning, please lead them back to any of the questions above and ask to return to the question.This Q allows participants to give feedback to the project and helps identify gaps/challenges, but we know many of the problems already and do not want to let this dominate/ mislead the main focus of the interview.Use this opportunity to increase the depth of any previous answers by probing and relating this question to any other points informants raise – if/when appropriate.QAF: Rel3, Rel5, Rel5, Rel7, Cre1, Leg3
18. What would have happened in young people’s engagement in post-conflict truth telling/transitional justice processes if this research had not been conducted?	• Probe to clarify if needed (the role of the project in improving collaboration, social networks, participation, engagement, etc.)	Testing “zero hypothesis”.Using a different angle to understand the true influence of ICRAF by asking what would be different had ICRAF not done its work.QAF: Eff4