Reflections on implementing the evaluative framework (EF)
| Evaluation framework (EF) – interview question | Response Recovery Co. | Response Social Care Co. | Comment |
|---|---|---|---|
| 1. What were your personal expectations of what the story project would deliver in terms of your own learning? (Prompts: In what way were these realised? In what way were they different?) | Leads and those involved with the design of the project responded easily; OTTs did not | Cohort participants including partnership organisations responded easily. Not asked of strategic lead | Keep in the generic evaluation framework |
| 2. What has been your experience of using what you learnt in your everyday environment? (Prompts: new skills, understanding, or behaviours) | Leads and those involved with the design of the project responded easily; OTTs did not | Cohort participants including partnership organisations responded easily. Not asked of strategic lead | Keep in the generic evaluation framework |
| 3. What has particularly enabled you to use this learning in your workplace? (Prompts: Opportunities? Particular support?) | Leads and those involved with the design of the project responded easily; OTTs did not | Cohort participants including partnership organisations responded easily. Not asked of strategic lead | Contextual: use in evaluation framework for projects with skills training element |
| 4. What has made it difficult to use this learning in your workplace? (Prompts: Obstacles? Lack of opportunities? Culture?) | Leads and those involved with the design of the project responded easily; OTTs did not | Cohort participants responded easily. Not asked of strategic lead | Contextual: use in evaluation framework for projects with skills training element |
| 5. Looking back at the last 6 months, i.e. the duration of the current story project, what has been the most significant change for you in your own work as a result of this project? (Prompts: Behaviours? Practices? Team work?) | Leads and those involved with the design of the project responded easily; OTTs made partial response | Cohort participants responded easily. Not asked of strategic lead | Keep in the generic evaluation framework |
| 6. What were your initial expectations of what the story project would deliver in terms of organisational benefits? (Prompts: In what way were these realised? In what way were they different?) | Leads and those involved with the design of the project responded easily; OTTs did not | Strategic lead responded easily; partnership organisation member did not | Contextual: use in evaluation framework for projects delivered at management or leadership level |
| 7. What have been the actual outcomes and benefits to the organisation? (Prompts: Efficiency. Budgetary. Knowledge. Partnership working) | Leads and those involved with the design of the project responded easily; OTTs did not | Strategic lead responded easily; partnership organisation member did not. Project manager had difficulty responding. | Contextual: use in evaluation framework for projects delivered at management or leadership level; review sample selection |
| 8. Looking back at the last [XX] months, i.e. the duration of the current story project, what do you think has been the most significant change in the organisation’s service delivery, as a result of this project? (Prompts: Better delivery of Recovery services. Better teamwork. Better partnership working) | Leads and those involved with the design of the project responded easily; OTTs did not | Strategic lead responded easily; partnership organisation member did not. Some difficulty in responding from original project manager (see above) | Contextual: use in evaluation framework for projects delivered at management or leadership level; review sample selection |
| 9. Looking back at the last [XX] months, i.e. the duration of the current story project, what has been the most significant change for your clients (and/or stakeholders and partnership organisations)? (Prompts: Social return on investment. Improvements in well-being or confidence. Better client/organisation relationships. Better take-up of services) | Leads and those involved with the design of the project responded easily; OTTs made partial response | Strategic lead responded easily; partnership organisation member did not. Some difficulty in responding from original project manager (see above) | Keep in generic evaluation framework; review sample selection |
| 10. Looking ahead, what are your recommendations to your organisation regarding future story-based projects? (Prompts: More workshops? More training? Sustainability & Improvements? Less/None?) | All interviewees responded easily | All interviewees responded easily | Keep in the generic evaluation framework |
| Evaluation framework (EF) – interview question | Response Recovery Co. | Response Social Care Co. | Comment |
|---|---|---|---|
| 1. What were your personal expectations of what the story project would deliver in terms of your own learning? (Prompts: In what way were these realised? In what way were they different?) | Leads and those involved with the design of the project responded easily; OTTs did not | Cohort participants including partnership organisations responded easily. Not asked of strategic lead | Keep in the generic evaluation framework |
| 2. What has been your experience of using what you learnt in your everyday environment? (Prompts: new skills, understanding, or behaviours) | Leads and those involved with the design of the project responded easily; OTTs did not | Cohort participants including partnership organisations responded easily. Not asked of strategic lead | Keep in the generic evaluation framework |
| 3. What has particularly enabled you to use this learning in your workplace? (Prompts: Opportunities? Particular support?) | Leads and those involved with the design of the project responded easily; OTTs did not | Cohort participants including partnership organisations responded easily. Not asked of strategic lead | Contextual: use in evaluation framework for projects with skills training element |
| 4. What has made it difficult to use this learning in your workplace? (Prompts: Obstacles? Lack of opportunities? Culture?) | Leads and those involved with the design of the project responded easily; OTTs did not | Cohort participants responded easily. Not asked of strategic lead | Contextual: use in evaluation framework for projects with skills training element |
| 5. Looking back at the last 6 months, i.e. the duration of the current story project, what has been the most significant change for you in your own work as a result of this project? (Prompts: Behaviours? Practices? Team work?) | Leads and those involved with the design of the project responded easily; OTTs made partial response | Cohort participants responded easily. Not asked of strategic lead | Keep in the generic evaluation framework |
| 6. What were your initial expectations of what the story project would deliver in terms of organisational benefits? (Prompts: In what way were these realised? In what way were they different?) | Leads and those involved with the design of the project responded easily; OTTs did not | Strategic lead responded easily; partnership organisation member did not | Contextual: use in evaluation framework for projects delivered at management or leadership level |
| 7. What have been the actual outcomes and benefits to the organisation? (Prompts: Efficiency. Budgetary. Knowledge. Partnership working) | Leads and those involved with the design of the project responded easily; OTTs did not | Strategic lead responded easily; partnership organisation member did not. Project manager had difficulty responding. | Contextual: use in evaluation framework for projects delivered at management or leadership level; review sample selection |
| 8. Looking back at the last [XX] months, i.e. the duration of the current story project, what do you think has been the most significant change in the organisation’s service delivery, as a result of this project? (Prompts: Better delivery of Recovery services. Better teamwork. Better partnership working) | Leads and those involved with the design of the project responded easily; OTTs did not | Strategic lead responded easily; partnership organisation member did not. Some difficulty in responding from original project manager (see above) | Contextual: use in evaluation framework for projects delivered at management or leadership level; review sample selection |
| 9. Looking back at the last [XX] months, i.e. the duration of the current story project, what has been the most significant change for your clients (and/or stakeholders and partnership organisations)? (Prompts: Social return on investment. Improvements in well-being or confidence. Better client/organisation relationships. Better take-up of services) | Leads and those involved with the design of the project responded easily; OTTs made partial response | Strategic lead responded easily; partnership organisation member did not. Some difficulty in responding from original project manager (see above) | Keep in generic evaluation framework; review sample selection |
| 10. Looking ahead, what are your recommendations to your organisation regarding future story-based projects? (Prompts: More workshops? More training? Sustainability & Improvements? Less/None?) | All interviewees responded easily | All interviewees responded easily | Keep in the generic evaluation framework |