Table A10.

Individual evaluator and average scoring of TTP for all QAF criteria, with justifications for the score allocated (Claus et al., 2020)

PrincipleCriteriaE1E2E3E4Avg.Justification/Comments
RelevanceClearly defined socio-ecological context22222Dissertation provides clear and full description of the country and problem context; reviews background information from multiple dimensions (historical, political, cultural, international regulatory standards/protocols, psychological, etc.); clear identification of entry points (literature gap, practice gap)
 Socially relevant research problem22222Research problem is well-aligned with the Ugandan context; informants corroborate entry points and relevance/value of study; dissertation reflects consideration of the practical application of the research activities and outputs in multiple dimensions
 Engagement with problem context22222Lead researcher had previous volunteer experience in Uganda, ten years of professional experience with IICRD, and experience with the Sierra Leone Truth and Reconciliation Commission (the first commission to systematically involve young people as active participants); hired local researchers which expanded influence potential; networked with local organizations working on the topic (RLP, TPO Uganda) which expanded influence potential; informants believed the research team engaged ethically and appropriately within the problem context
 Explicit theory of change22222Dissertation explicitly identifies and documents the main intended outcomes, how they would be realized, and relevant progress markers
 Relevant research objective and design22222Decisions on research design were guided by ‘do no harm’ and with the interests of young people at forefront; both objectives and design were co-generated within the research team; activities were piloted and revised based on feedback from YPRA participants; PAR-driven research design was relevant and appropriate to the problem context and objectives of the study to promote the authentic voice, needs, and perspectives of young people and build their capacities around safe and meaningful engagement in transitional justice processes
 Appropriate project implementation22222Participatory execution was well-aligned to address potential problems within the problem context and satisfy the objectives; all activities were piloted with YPRA participants and revised based on feedback; diverse representation of young people affected by the conflict and partner organizations were engaged throughout the research process; informants viewed project engagement to be culturally sensitive and meaningful
 Effective communication22222Research strategically identified and communicated with relevant actors (young people, partner organizations, local to national target audiences); strong internal dialogue processes between members of the research team; research activity communication was thought to be clear and appropriate; communication of research activities used multiple strategies to meet diverse needs (use of speech, writing, drawing, and both language and sign interpreters); verification workshops were held with participants to ensure understanding and accuracy of findings; planned a follow-up ‘evaluation’ workshop with participants post-project; dissertation is well communicated; produced diverse research outputs for multiple audiences (e.g. conferences, news articles, newsletters, articles, SSHRC story-telling, reports, etc.)
CredibilityBroad preparation22222Literature review is extensive and comprehensive; external evaluator noted “This dissertation, which interweaves elements of transitional justice and truth-telling, psychosocial support, child protection, and research ethics, is an excellent piece of research that makes highly significant contributions to both knowledge and practice” (Doc6) and “The document makes appropriate and deep contact with the extant literature” (Doc6); dissertation expressly addresses challenge that PAR is not a theoretical framework, provides justification, and demonstrates true understanding of the approach
 Clear research problem definition22222Research gaps are identified; research problem is clearly stated in the dissertation, grounded within the literature, and explicitly connected to the context
 Clear research question22222Research questions clearly presented in the dissertation; transparency is given for the addition of a research question informed by participants; research questions are justified and grounded in the problem context
 Objectives stated and met22222Objectives are clearly stated in the dissertation; objectives have been met
 Feasible research project22222Project was supported by sufficient funding (SSHRC [Joseph-Armand Bombardier Canada Graduate Scholarship, Michael Smith Foreign Study Supplement], IDRC); research team and partner resources were thought to be appropriate; research activities/tools were flexible to adapt to different participants’ needs; some young people participants wished there had been more time for the activities; informants’ suggestions for what could have been done differently fell outside the bounds of a doctoral research project
 Adequate competencies22222Research team demonstrated a diverse and complementary set of competencies to the project; lead researcher had extensive research experience; Ugandan members ensured cultural appropriateness and had relevant research experience; team sought support for language and signing resources where appropriate; lead researcher offered training to co-researchers and research assistants; even with research assistant turnover mid-project, the position was refilled
 Research approach fits purpose22222Lead researcher brought in principled learning from DSocSci course, applying and reflecting multiple ways of knowing into the project design and outputs; utilized and adapted participatory methods/tools previously used at IICRD with input from partner organizations (RLP, TPO Uganda) and YPRA pilot; participatory approach of the research bridged the gap of young people’s voices in a transitional justice process which is normally regulated by adults; reflection on paradoxes/conflicts were brief, but present (re: inviting young people to share how they wished to share their experiences without coercion to actually share their stories); approach is justified in documentation and informants’ reflections demonstrate support for the approach taken considering the context and objectives of the research
 Appropriate method22222Methods description is extensive and transparent in the dissertation; methods are fit to purpose; methods were tested with the YPRA pilot, representing the groups engaged in the main study; tools/methods selected were adaptable based on pilot participant and partner feedback/input, and reflect participatory principles central to the approach; activities were adapted to fit different participants’ needs and clearly justified
 Clearly presented argument22222Results are clearly presented in the dissertation and throughout other research outputs; results presented in a logical manner, sharing young people’s perspectives by the different groups (BIC, FA, DBW, ID, etc.) – these were thought to be important findings to inform how to engage with different groups; clear demonstration of connection between evidence/results (excerpts and direct quotations), recommendations, and conclusions; alternative explanations are not explored (however, they do not apply to this type of research)
 Transferability and generalizability of the findings22222Dissertation reflects on the extent of the generalizability of the findings for other contexts; informants believed the research process was highly transferable and an exemplary case of PAR; evidence of transfer of research approach to other contexts (e.g. RRU Resilience by Design Lab, lead researcher’s current research focus in cancer and healthcare context, participatory Ugandan curriculum development on post-conflict processes, external advisor’s community-based approach and inclusion of vulnerable voices, RRU’s School of Leadership Studies, supervisor’s work on the Colombian truth and reconciliation commission, supervisor’s working partnerships with First Nations in British Columbia); while findings are case-specific, many of the principles/recommendations are thought to be transferable by informants
 Limitations stated22222Discusses shortcomings of the methods (e.g. PAR, sample size); discusses challenges/dilemmas encountered (e.g. research barriers, geography, cultural barriers, compensation to participants, managing expectations, delays); discussion on limitations of results are limited (e.g. representativeness), but there are indications from researcher interviews and internal project documents that limitations were accounted for on an on-going basis
 Ongoing reflexivity and monitoring22222Inclusion of processes for reflection are explained (e.g. YPRA pilot, daily research team reflection [summary reflection form], journaling, post-project ‘evaluation’ workshop); some reflections and changes made based on those processes are presented transparently in the dissertation (e.g. added research question, adapted methodology, expanded inclusion of young people’s groups and ages, pursuit of new opportunities); summary reflection form well designed to include aspects for what the research team learned, what could be done differently, and what follow-up could look like
LegitimacyDisclosure of perspective22222Dissertation references funding sources, partners, and researcher positionality; dissertation discusses selection and justification for partnership and collaborations; dissertation discusses lead researcher and research team biases (gender, ethnicity, socio-economic background, education, privilege); dissertation discusses power dynamics and consideration for marginalization
 Effective collaboration22222Internal project documents outline roles and responsibilities of the research team and partners; egalitarian decision-making processes within the research team; members of the research team reflected on the interpersonal relationships and collaboration experience positively; effective collaboration with partners and young people was central to the participatory nature of the project; lead researcher created genuine opportunities for other members of the research team to grow and gain professional experience (e.g. co-authoring of articles, presenting at conferences); partnership with RLP and TPO Uganda reflected on positively by research team and members of those organizations
 Genuine and explicit inclusion22222Genuine engagement of young people’s voices and ideas into the project design (e.g. YPRA pilot) and results (participants); co-researchers/research assistants felt involved and invested in the project; co-researchers, partners, and supervisory committee felt their input was included throughout research process; values of participation, ethics, and inclusion clearly upheld throughout the research process; design reflected conscientious inclusion of gender dynamics, diverse groups’ needs, power dynamics, ‘do no harm’, and cultural appropriateness; built in opportunities to verify results with participants to ensure their voices were accurately represented; informants reflected on lead researcher’s social capital, orientation to teamwork, and sincerity
 Research is ethical22222Project received ethical approval by RRU Research Ethics Board; project submitted proposal to the Ugandan National Council for Science and Technology, and received approval; approval also received from three local government districts in Uganda; research team practiced informed and ongoing consent with their participants throughout the research process; constant researcher reflection on principles of ‘do no harm’ and Right of the Child were at the forefront of decision-making and research practice; participants’ reflections on their engagement demonstrated the research was conducted ethically (e.g. “researchers were polite and good”, “confidentiality is kept”); follow-up verification of findings and ‘evaluation’ workshop were reflective of ethical research practice; practitioner perceptions indicate the research upheld the dignity of the participants
EffectivenessResearch builds social capacity22222Participants’ reflections demonstrate learning occurred regarding the Ugandan conflict, transitional justice processes, how to engage/share with others, and sources of support/help; other researchers felt the research was valuable for the learning and opportunities given to participants; participants gained courage to share and express themselves, which they previously did not have; partners learned of actor gaps/opportunities in their work and how they could engage young people; practitioners believed the project made important knowledge contributions; co-researchers developed research skills, professional capacities, and networks; project influenced changes in perspectives/recognition amongst participants, the research team, partners, and practitioners (IICRD); informants spoke to positive perceptions of participants and partners they felt were influenced by the research personally and professionally; some practitioners were inspired by the project to reflect on or change how they approached their own work
 Contribution to knowledge22222Research made a contribution to the knowledge base; built participants’ knowledge in the topic of transitional justice; built partners’ knowledge around persistent issues for their practice; some contribution made to the development to the Kampala Recommendations (some of the project’s findings are reflected in that document), though they have not been taken up or applied widely; shared findings through a webinar, an article in a special issue, relevant conferences, and the SSHRC Story-tellers competition to reach a wider audience than the dissertation; lead researcher made knowledge contributions to the Sri Lankan Truth and Reconciliation Commission; practitioners viewed research to make a significant contribution that was missing from the literature
 Practical application22222Contributions shared and applied in Kampala Recommendations, partners’ practices (e.g. involvement of young people, Kitgum Festival, documentary), Ugandan Transitional Justice Policy, Ugandan curriculum development, Sri Lankan Truth and Reconciliation Commission; lead researcher and co-researchers continue to adapt and apply approaches in their work (e.g. RRU Resilience by Design Lab, RRU School of Leadership Studies, NMPDC, working with young people); members of advisory committee research influenced by project (e.g. Colombian Truth and Reconciliation Commission, reconciliation processes with First Nations in British Columbia)
 Significant outcome22222Some participants felt the project changed their life; practitioner and researcher informants perceived the project supported and empowered the young people that participated; high potential that contributions supported positive social change through the Sri Lankan Truth and Reconciliation Commission; high potential that contributions to Ugandan Transitional Justice Policy will take effect; how co-researchers and partner organizations are engaging within the local context will have positive social implications
[ViewLarge]
This Feature Is Available To Subscribers Only

or Create an Account

Close subscription notice
Close access options