Individual evaluator and average scoring of SNDP for all QAF criteria, with justifications for the score allocated
| Principle | Criteria | E1 | E2 | E3 | E4 | Avg. | Justification/Comments |
|---|---|---|---|---|---|---|---|
| Relevance | Clearly defined socio-ecological context | 1 | 2 | 1 | 1 | 1.25 | Dissertation provides a historical, geographical, socio-economic, and socio-cultural description of the problem context; dissertation outlines the complexity of WASH in the Niger Delta and explains compounding factors; identification of academic entry points could have been stronger; dissertation does not explain fully the community, NGO, and policy contexts of the WASH sector in the Niger Delta (would have strengthened the justification for why the research is needed) |
| Socially relevant research problem | 2 | 2 | 2 | 2 | 2 | Academic and practical gaps are identified and discussed (e.g. negative spillover effects of poor sanitation, appropriateness of WASH technologies for riverine systems are unknown, siloization of Nigeria’s WASH sector); research problem is aligned with Millennium Development Goal 7; informants reflect on relevance and value of study; practical application of the findings are considered and discussed (e.g. PIND) | |
| Engagement with problem context | 2 | 2 | 2 | 1 | 1.75 | PI interacted sufficiently with the problem context to gain a breadth and depth of understanding (via personal experience as a WASH practitioner in Africa, fieldwork engaged a range of system actors); PI was well positioned to influence the context (access to Nigerian WASH networks, working at PIND), but unclear the extent to which this was leveraged (e.g. problem context engagement versus information extraction) | |
| Explicit theory of change | 1 | 1 | 1 | 1 | 1 | Not explicit or documented, but implicit in statements of hoped or intended changes; dissertation indicates opportunities to influence change via PIND or other mechanisms, though this is briefly discussed | |
| Relevant research objective and design | 1 | 1 | 1 | 1 | 1 | Singular objective is weak (lacks specificity) and reads more as a statement of interest; research design is relevant and appropriate to the problem context and exploratory nature of the project; research accounts for and accommodates community needs/values in the design of the approach on sensitive and taboo topic | |
| Appropriate project implementation | 1 | 2 | 1 | 1 | 1.25 | Exploratory approach appropriate considering the scope of the state objective; PI had ongoing engagement with system actors; unclear how new knowledge or reflections made during the research process were incorporated (not documented) | |
| Effective communication | 1 | 1 | 1 | 1 | 1 | No discussion is given on how communications during the research process were planned; research advisory committee commended the PI’s communication of progress and feedback; multiple dissemination strategies were planned; communication of results was perceived to be accessible and well-organized by informants, though there was low awareness of results amongst informants and other actors | |
| Credibility | Broad preparation | 1 | 2 | 1 | 1 | 1.25 | Documentation conveys understanding from multiple disciplines, though the integration this knowledge could have been strengthened; various theories and models of behaviour change were reviewed; in-depth understanding of technological options available for the Niger Delta were reviewed and described; the research does not draw on enough previous research from other riverine contexts (only mentioned in passing); engagement with knowledge of former interventions in the Niger Delta was missing |
| Clear research problem definition | 1 | 1 | 1 | 1 | 1 | The research problem is stated in the dissertation, though broad; the grounding of the research problem in academic literature is vague | |
| Clear research question | 1 | 1 | 1 | 1 | 1 | The research question is stated in the dissertation; appropriateness of the research question was questioned owing to its broadness; some informants critiqued the framing of the research question, feeling that it did not lend well for the project to contribute to a growth of understanding in any one area | |
| Objectives stated and met | 1 | 1 | 1 | 1 | 1 | A singular objective is stated and appears to have been met; however, the objective is not logically or appropriately related to the problem context; a better formulation of objectives would have aided the structure and purpose of the project | |
| Feasible research project | 1 | 1 | 1 | 2 | 1.25 | The design and resources were appropriate to the objective as stated; the project was supported by sufficient funding (Mitacs Fellowship) for fieldwork and hiring of enumerators; PI drew on existing networks through PIND and WASH practitioners; resilience of the design to accommodate opportunities or challenges is not discussed | |
| Adequate competencies | 2 | 2 | 2 | 2 | 2 | The PI had extensive prior field experience and knowledge of Nigeria and the WASH sector; enumerators were given training to conduct the survey; informants commented on the PI’s passion for the work | |
| Research approach fits purpose | 1 | 1 | 1 | 1 | 1 | Exploratory approach fits the stated objective; rationale is given for the approach taken to collect data from the communities (accounting for stigma associated with the topic); there is clear discussion for the inclusion of methodologies, but the selection of disciplines and how they were integrated is not discussed; no discussion of paradoxes or conflicts | |
| Appropriate method | 1 | 2 | 1 | 1 | 1.25 | Mixed methods are clearly described, systematic, and fit to the exploratory nature of the research; however, missed opportunity in the analysis to fully leverage the data collected indicates inappropriateness of the methods | |
| Clearly presented argument | 1 | 1 | 1 | 1 | 1 | Results are clearly presented in both narrative and diagrams; analyses do not demonstrate synthesis of results, instead reading as a list of anecdotes and informant opinions; missed opportunity to draw correlations between the data for a strong argument; the recommendations are not logically connected to the results; alternative explanations are not explored | |
| Transferability and generalizability of the findings | 1 | 1 | 1 | 1 | 1 | Findings are too specific to be transferable; informants thought the research was transferable, as sanitation is a global issue; methods would be transferable, but this is not discussed | |
| Limitations stated | 0 | 0 | 0 | 0 | 0 | Only one brief section of the dissertation acknowledges limited data availability and difficulty in corroborating self-reported qualitative research; limitations are not meaningfully discussed in terms of the implications of results; most important limitations are not addressed | |
| Ongoing reflexivity and monitoring | 1 | 1 | 1 | 0 | 0.75 | Dissertation mentions the importance of reflexivity, but it is not clear the extent to which or how reflection was done; one informant noted the survey underwent significant re-design, but this is not documented in the dissertation | |
| Legitimacy | Disclosure of perspective | 0 | 0 | 1 | 0 | 0.25 | Dissertation indicates an awareness of personal biases, but these biases are not openly explained or discussed in terms of the implications for the research; researcher positionality is not discussed; Mitacs funding is acknowledged; partnership with PIND is noted, but connection with Shell and Chevron are not disclosed; notes some participants were known to the PI from own work in the region |
| Effective collaboration | 1 | 0 | 1 | 0 | 0.5 | The research is not a collaborative piece of work (evidence indicates extractive relationship); there was scope for the research to be more collaborative through the PI’s professional connections to PIND and other WASH practitioners (impression that the PI separated the doctoral work from PIND work); members of the advisory committee reflected positively on their relationships with the PI | |
| Genuine and explicit inclusion | 1 | 1 | 1 | 1 | 1 | A range of system actors were involved in the research, but specifics of their inclusion is not described; steps were taken to engage communities in an appropriate way (being sensitive to their needs and values), but not clear if done for all system actors; indications that the PI made efforts to establish a positive relationship with participants to build trust; creation of the WhatsApp group facilitated a platform for cross-pollination of ideas, perspectives, and experiences amongst WASH practitioners | |
| Research is ethical | 1 | 2 | 2 | 2 | 1.75 | Project received ethical approval by RRU Research Ethics Board; anonymity of participants is preserved; consideration is given on how to ethically engage communities on a sensitive or taboo topic; ethical importance of reflexivity is mentioned | |
| Effectiveness | Research builds social capacity | 1 | 1 | 1 | 1 | 1 | Research capacity-building of the PI was high; enumerators received training, but their research capacity is not considered; scope for more intentional capacity-building via the project was a missed opportunity; indications from informants that they think more deeply about the topic and related issues as a result of the research (i.e. how to support development in the Niger Delta); enhanced knowledge-sharing and learning occurring in the Nigerian WASH sector, but unclear if this is a contribution of the project |
| Contribution to knowledge | 1 | 1 | 1 | 1 | 1 | Contributed to PI’s knowledge; scope for more was a missed opportunity; research not thought to have contributed to new knowledge, but supported bridging connections in existing knowledge and expanding on what is already known | |
| Practical application | 1 | 1 | 1 | 1 | 1 | No innovations were developed from the research; recommendations were intended for uptake, but unclear to what extent this happened; there are some indications of elements taken up by the WASH community, but unclear how and if connected to the project | |
| Significant outcome | 2 | 2 | 1 | 1 | 1.5 | Six out of ten end-of-project outcomes were fully or partially realized, with five indicating clear contribution of the project; potential for more in the future (too early to assess social/economic/environmental benefits), but likely this will result from other processes and interventions in the context |
| Principle | Criteria | E1 | E2 | E3 | E4 | Avg. | Justification/Comments |
|---|---|---|---|---|---|---|---|
| Relevance | Clearly defined socio-ecological context | 1 | 2 | 1 | 1 | 1.25 | Dissertation provides a historical, geographical, socio-economic, and socio-cultural description of the problem context; dissertation outlines the complexity of WASH in the Niger Delta and explains compounding factors; identification of academic entry points could have been stronger; dissertation does not explain fully the community, NGO, and policy contexts of the WASH sector in the Niger Delta (would have strengthened the justification for why the research is needed) |
| Socially relevant research problem | 2 | 2 | 2 | 2 | 2 | Academic and practical gaps are identified and discussed (e.g. negative spillover effects of poor sanitation, appropriateness of WASH technologies for riverine systems are unknown, siloization of Nigeria’s WASH sector); research problem is aligned with Millennium Development Goal 7; informants reflect on relevance and value of study; practical application of the findings are considered and discussed (e.g. PIND) | |
| Engagement with problem context | 2 | 2 | 2 | 1 | 1.75 | PI interacted sufficiently with the problem context to gain a breadth and depth of understanding (via personal experience as a WASH practitioner in Africa, fieldwork engaged a range of system actors); PI was well positioned to influence the context (access to Nigerian WASH networks, working at PIND), but unclear the extent to which this was leveraged (e.g. problem context engagement versus information extraction) | |
| Explicit theory of change | 1 | 1 | 1 | 1 | 1 | Not explicit or documented, but implicit in statements of hoped or intended changes; dissertation indicates opportunities to influence change via PIND or other mechanisms, though this is briefly discussed | |
| Relevant research objective and design | 1 | 1 | 1 | 1 | 1 | Singular objective is weak (lacks specificity) and reads more as a statement of interest; research design is relevant and appropriate to the problem context and exploratory nature of the project; research accounts for and accommodates community needs/values in the design of the approach on sensitive and taboo topic | |
| Appropriate project implementation | 1 | 2 | 1 | 1 | 1.25 | Exploratory approach appropriate considering the scope of the state objective; PI had ongoing engagement with system actors; unclear how new knowledge or reflections made during the research process were incorporated (not documented) | |
| Effective communication | 1 | 1 | 1 | 1 | 1 | No discussion is given on how communications during the research process were planned; research advisory committee commended the PI’s communication of progress and feedback; multiple dissemination strategies were planned; communication of results was perceived to be accessible and well-organized by informants, though there was low awareness of results amongst informants and other actors | |
| Credibility | Broad preparation | 1 | 2 | 1 | 1 | 1.25 | Documentation conveys understanding from multiple disciplines, though the integration this knowledge could have been strengthened; various theories and models of behaviour change were reviewed; in-depth understanding of technological options available for the Niger Delta were reviewed and described; the research does not draw on enough previous research from other riverine contexts (only mentioned in passing); engagement with knowledge of former interventions in the Niger Delta was missing |
| Clear research problem definition | 1 | 1 | 1 | 1 | 1 | The research problem is stated in the dissertation, though broad; the grounding of the research problem in academic literature is vague | |
| Clear research question | 1 | 1 | 1 | 1 | 1 | The research question is stated in the dissertation; appropriateness of the research question was questioned owing to its broadness; some informants critiqued the framing of the research question, feeling that it did not lend well for the project to contribute to a growth of understanding in any one area | |
| Objectives stated and met | 1 | 1 | 1 | 1 | 1 | A singular objective is stated and appears to have been met; however, the objective is not logically or appropriately related to the problem context; a better formulation of objectives would have aided the structure and purpose of the project | |
| Feasible research project | 1 | 1 | 1 | 2 | 1.25 | The design and resources were appropriate to the objective as stated; the project was supported by sufficient funding (Mitacs Fellowship) for fieldwork and hiring of enumerators; PI drew on existing networks through PIND and WASH practitioners; resilience of the design to accommodate opportunities or challenges is not discussed | |
| Adequate competencies | 2 | 2 | 2 | 2 | 2 | The PI had extensive prior field experience and knowledge of Nigeria and the WASH sector; enumerators were given training to conduct the survey; informants commented on the PI’s passion for the work | |
| Research approach fits purpose | 1 | 1 | 1 | 1 | 1 | Exploratory approach fits the stated objective; rationale is given for the approach taken to collect data from the communities (accounting for stigma associated with the topic); there is clear discussion for the inclusion of methodologies, but the selection of disciplines and how they were integrated is not discussed; no discussion of paradoxes or conflicts | |
| Appropriate method | 1 | 2 | 1 | 1 | 1.25 | Mixed methods are clearly described, systematic, and fit to the exploratory nature of the research; however, missed opportunity in the analysis to fully leverage the data collected indicates inappropriateness of the methods | |
| Clearly presented argument | 1 | 1 | 1 | 1 | 1 | Results are clearly presented in both narrative and diagrams; analyses do not demonstrate synthesis of results, instead reading as a list of anecdotes and informant opinions; missed opportunity to draw correlations between the data for a strong argument; the recommendations are not logically connected to the results; alternative explanations are not explored | |
| Transferability and generalizability of the findings | 1 | 1 | 1 | 1 | 1 | Findings are too specific to be transferable; informants thought the research was transferable, as sanitation is a global issue; methods would be transferable, but this is not discussed | |
| Limitations stated | 0 | 0 | 0 | 0 | 0 | Only one brief section of the dissertation acknowledges limited data availability and difficulty in corroborating self-reported qualitative research; limitations are not meaningfully discussed in terms of the implications of results; most important limitations are not addressed | |
| Ongoing reflexivity and monitoring | 1 | 1 | 1 | 0 | 0.75 | Dissertation mentions the importance of reflexivity, but it is not clear the extent to which or how reflection was done; one informant noted the survey underwent significant re-design, but this is not documented in the dissertation | |
| Legitimacy | Disclosure of perspective | 0 | 0 | 1 | 0 | 0.25 | Dissertation indicates an awareness of personal biases, but these biases are not openly explained or discussed in terms of the implications for the research; researcher positionality is not discussed; Mitacs funding is acknowledged; partnership with PIND is noted, but connection with Shell and Chevron are not disclosed; notes some participants were known to the PI from own work in the region |
| Effective collaboration | 1 | 0 | 1 | 0 | 0.5 | The research is not a collaborative piece of work (evidence indicates extractive relationship); there was scope for the research to be more collaborative through the PI’s professional connections to PIND and other WASH practitioners (impression that the PI separated the doctoral work from PIND work); members of the advisory committee reflected positively on their relationships with the PI | |
| Genuine and explicit inclusion | 1 | 1 | 1 | 1 | 1 | A range of system actors were involved in the research, but specifics of their inclusion is not described; steps were taken to engage communities in an appropriate way (being sensitive to their needs and values), but not clear if done for all system actors; indications that the PI made efforts to establish a positive relationship with participants to build trust; creation of the WhatsApp group facilitated a platform for cross-pollination of ideas, perspectives, and experiences amongst WASH practitioners | |
| Research is ethical | 1 | 2 | 2 | 2 | 1.75 | Project received ethical approval by RRU Research Ethics Board; anonymity of participants is preserved; consideration is given on how to ethically engage communities on a sensitive or taboo topic; ethical importance of reflexivity is mentioned | |
| Effectiveness | Research builds social capacity | 1 | 1 | 1 | 1 | 1 | Research capacity-building of the PI was high; enumerators received training, but their research capacity is not considered; scope for more intentional capacity-building via the project was a missed opportunity; indications from informants that they think more deeply about the topic and related issues as a result of the research (i.e. how to support development in the Niger Delta); enhanced knowledge-sharing and learning occurring in the Nigerian WASH sector, but unclear if this is a contribution of the project |
| Contribution to knowledge | 1 | 1 | 1 | 1 | 1 | Contributed to PI’s knowledge; scope for more was a missed opportunity; research not thought to have contributed to new knowledge, but supported bridging connections in existing knowledge and expanding on what is already known | |
| Practical application | 1 | 1 | 1 | 1 | 1 | No innovations were developed from the research; recommendations were intended for uptake, but unclear to what extent this happened; there are some indications of elements taken up by the WASH community, but unclear how and if connected to the project | |
| Significant outcome | 2 | 2 | 1 | 1 | 1.5 | Six out of ten end-of-project outcomes were fully or partially realized, with five indicating clear contribution of the project; potential for more in the future (too early to assess social/economic/environmental benefits), but likely this will result from other processes and interventions in the context |
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.