An Integrated Knowledge Translation Approach to Developing A Story-Based Positive Youth Development Program in Sport: The 1616 Program Free

The 1616 Program was instigated by Brandy and Andrew Ladd, the founders of the Ladd Foundation. Andrew Ladd is a current professional hockey player in the National Hockey League (NHL), and along with his partner Brandy, they hoped to promote the healthy development of young hockey players by providing free online programming. The name of the program stems from Andrew’s playing number (i.e., 16), the fact that the final program will be 16 weeks in duration, and importantly, that the year 1616 is credited with the introduction of the term ‘buffalo’ to describe the American Bison. The theme of the 1616 Program is to develop a ‘Buffalo Mindset’ as, unlike most animals, buffalo have been described as banding together and moving towards and through a storm, using their herd for support to overcome challenges more quickly and effectively. Such a mantra mimics the importance of identity, belonging, and support that are critical for ideal youth experiences and healthy development in team sport (e.g., Allen, 2006; Bruner et al., 2017).

Given the complex and iterative nature of engaging in an iKT approach, the following section serves to introduce the partners that were actively involved in the development of the program, the iKT principles adhered to for collaborative partnerships, and the guiding framework for program development. We then describe in detail the process undertaken as it aligned with the guiding iKT principles and framework.

The Ladd Foundation is a charitable organization set on promoting developmentally rich opportunities to North American youth (10 to 12 years of age) involved in ice hockey. Notably, they sought to promote healthy sport experiences by capitalizing on inspirational stories from current elite athletes. To best achieve this objective, they partnered with Impact Society (www.impactsociety.com), a research team, and several media production companies to develop the 1616 Program. Given that optimal iKT involves the process of influencing decision-making by getting the right information, to the right people, in the right format, at the right time (Levin, 2008), the overarching aim of the 1616 Program was to translate PYD and character development-related information to relevant ice hockey end-users (i.e., children, parents, coaches). In this regard, Impact Society was recruited because of their expertise in story-based development and delivery of educational curricula for young people. The research group was identified during the initial market research and invited to join the partnership to ensure that all partners had a comprehensive grasp of available research and that the programming would be evidence-informed. Together, the Ladd Foundation, Impact Society, and the research team represent the ‘curriculum committee’ tasked with developing the content for the program. To produce video and digital media content that could be delivered in an online, virtual medium in an engaging and professional manner, Anthem Creative (https://anthemcreative.ca/), Banner (https://www.banner.tv/), and The Post Game (http://www.thepostgame.com/), were recruited to form the ‘creative committee’ alongside representation from the Ladd Foundation. With Meredith Wolf serving as executive director, these committees worked collaboratively to develop the proposed programming.

The development of the 1616 Program has represented an iKT process from partnership inception. An important feature of iKT work is the integral involvement of major decision makers—in this instance, the Ladd Foundation—in every aspect of the design, development, and implementation of a program (Jull et al., 2017). To ensure a positive working relationship that resulted in programming best informed by the literature, while also satisfying the desires of those with the authority to enact change, we followed the guiding iKT principles put forth by Gainforth and colleagues (2020). A detailed description of these guiding principles can be found elsewhere (Gainforth et al., 2020); however, we generally (a) sought to establish relationships based on trust and respect, (b) prioritized open, honest, and responsive communication, (c) recognized that every partner was selected for specific expertise and thus, shared decision-making responsibilities, (d) ensured that all partners were flexible and receptive to tailoring the research, production, and delivery components of the program based on pre-defined aims, the context, and practical and financial constraints, and (e) made sure that all partners benefited from involvement. Examples of these principles in practice are described throughout the subsequent sections.

To guide our iKT approach, we adhered to a systematic knowledge translation and implementation framework. The KTA framework is the most frequently adopted model to guide such work (Leggat et al., 2021; Strifler et al., 2018) and has been used in a sport context (e.g., Allan et al., 2021). Figure 1 depicts the KTA processes for creating and applying knowledge in real-world, practical settings (Graham et al., 2006). The framework guides the processes of knowledge creation, which involves using research findings to generate knowledge tools or products (see interior triangle), and the application of knowledge in a particular context (see outer circle components). To best describe our iKT approach, we summarize our efforts across six broad categories based on components of the KTA framework. Note that throughout the process, there is consistent re-engagement with the knowledge creation process (i.e., interior triangle) to ensure that what is put forth is evidence-informed to the greatest extent possible.

As depicted in Figure 1 Section A, we first needed to identify the need, determine the most relevant and evidence-informed knowledge, and propose a plan. We describe these stages as Conceptualizing the issue, Bringing in the research, and Establishing the plan. We then needed to create and adapt knowledge products based on the target context while ensuring that all objectives/desires from the various partners were satisfied (Figure 1, Sections B and C). We describe these stages as Making the program’s knowledge tools and Getting initial end-user feedback. Finally, we piloted the program to monitor its use and evaluate intended outcomes (Figure 1, Sections D, E, F, and G). We describe these stages as Pilot program implementation and evaluation.

What Happened. The Ladd Foundation wanted to actively promote quality sport experiences for young ice hockey players and therefore, conducted an initial needs assessment to better understand the context of interest. During Fall 2020/Winter 2021, they conducted market research for available youth development programming and as is recommended in the literature (e.g., Fernandez et al., 2019; Mattie et al., 2020), also engaged in numerous discussions with a convenience sample of current athletes, parents, and coaches to inquire about their experiences in ice hockey. Their preliminary impressions highlighted the negative developmental effects for young athletes of increasing professionalization and early specialization in youth sport. From their initial search and the targeted discussions with invested sport partners, the Ladd Foundation felt that available youth sport PYD-type programs were overly structured which detracted from physical activity and playing time, that they rarely included coherent components for all end-users involved in the sport (i.e., athletes, parents, and coaches), and that most required participation fees.

As a result, the initial ‘problem’ they sought to confront was the professionalization of youth sport by providing age-appropriate messaging emphasizing the potential for sport to be a rich avenue for healthy development. This needed to be done without over structuring and detracting from physically active practice and competition time for athletes. They also wanted to include parents and coaches as active agents in the experience without overburdening them by imposing additional requirements (e.g., training workshops). With these objectives and priorities at the forefront, the planning phase commenced by first selecting the partners who could best support program development and aid with identifying, reviewing, and selecting the appropriate bodies of knowledge for program content creation.

Key Considerations. To tackle the identified problem, the Ladd Foundation recognized the need to engage with multiple partners with a broad range of skills and expertise, including teams that focus on creation and development, production, and research. However, none of these organizations or individuals involved had ever worked together previously, so getting to know one another, understanding the diverse ranges of workstyles and implicit assumptions held by each (e.g., providing foundational knowledge of research, youth programming requirements, production processes), and developing productive communication channels and processes (e.g., standing Zoom calls, creation of workflow documents) represented necessary initial investments of time (and ongoing efforts throughout the project). The initiating partner needed to consider who to best engage with and ensure that all subsequent partners’ visions aligned with theirs as an organization. Indeed, emerging research emphasizes the necessity of considering numerous factors that promote mutually beneficial partnerships established to achieve common objectives (Melton et al., 2022). As such, when approaching a similar partnership, researchers and other key partners might ask themselves, ‘Whose expertise do we need as part of the team?’, ‘How can we create a cohesive working group across the different areas of expertise?’, and ‘What is the best process to maintain transparent communication between the different partners?’.

What Happened. The research team held an initial series of presentations/workshops for the partners to discuss the scope of the youth development literature. As a summary, the presentations began by introducing how much of the sport literature is grounded in Lerner’s (2000) PYD research and builds on Bronfenbrenner’s (1999) ecological systems theory. The research team described how these perspectives suggest that development is facilitated by the two-way interactions between individuals and their environment. They introduced the many ecological frameworks pertaining to PYD in sport (e.g., Cairney et al., 2018; Côté et al., 2020; Holt et al., 2018) and how they could be used in practical settings such as youth hockey. After much discussion, the Personal Assets Framework (PAF; Côté et al., 2020) resonated most with members of the Ladd Foundation because of how it conceptualized the determinants and outcomes of youth sport engagement over different time scales. The research team was then asked to present the sport and PYD literature that underpins the PAF to the Ladd Foundation Board Members as well as groups of end-users (i.e., coaches and parents) to elicit feedback and discuss how the conceptual framework aligned with their personal experiences. The objectives of these sessions were to ensure that the partners (i.e., Ladd Foundation, Impact Society) felt empowered to make informed program decisions and that the story creation process undertaken by Impact Society would be rooted in sport and PYD literature that was relevant and practical. Similarly, the partners were interested in hearing initial reactions from end-users when introduced to the research that would serve as the foundation for the program.

As a general overview of the PAF, the framework emphasizes the importance of three dynamic elements that interact to create a holistic understanding of the sport environment: (a) personal engagement in activities (e.g., sampling within and between sports; deliberate practice, deliberate play), (b) appropriateness of the setting (e.g., adapted sport structures; age/skill-based groupings), and (c) quality social dynamics (e.g., relationships between teammates, parents, coaches). An important feature of the PAF is the emphasis on how the dynamic interaction between an individual and their social and physical sport environments can promote personal asset development known as the 4Cs. Extending from Lerner’s (2006) initial 5Cs, Côté and colleagues (2010) proposed a sport-specific version of the 4Cs (Competence, Confidence, Connection, and Character). For the 1616 Program, the 4Cs represent the foundational assets indicative of PYD.

Key Considerations. Although the expectation that youth development and character education programs be evidence-informed and scientifically grounded has increased, the research literature remains daunting and inaccessible for most programming organizations. The research team was recruited not simply for their knowledge of relevant research on PYD in youth sport, but more importantly, to act as a conduit between the scientific literature and the needs of the larger program development group. As such, a key challenge was translating and communicating research evidence, concepts, and theories in ways that were both meaningful and actionable for the relevant partners. Similarly, scholars have noted the challenge of balancing researchers’ goals of designing quality studies and their partner’s practical objectives for supporting their specific program (e.g., Ettekal et al., 2017). In the case of the 1616 program, multiple ecological models of youth sport participation were considered and presented; however, the research team found they were most comfortable with and best able to communicate clearly and engagingly about the PAF and 4Cs frameworks—content with which they were intimately familiar. This was reinforced by the strong positive reception of the larger project team and external end-users (e.g., parents, coaches) during content presentations. Thus, researchers in similar partnerships might ask themselves: ‘How can I communicate key research concepts in ways that are most meaningful for my program partners?’ ‘What bodies of research do the program partners need to be introduced to?’ and ‘What relevant theoretical/conceptual models am I most comfortable working with and sharing to meet the project's objectives?’

What Happened. The Ladd Foundation had a vision for a PYD program with two necessary components: It would (a) involve inspirational stories delivered from current elite athletes to capture the imagination and engagement of youth, and (b) be free of charge to any participating teams or organizations. To these ends, some important decisions based on the 4Cs were required. All decisions were made collaboratively, and every partner was given an opportunity to provide insight prior to moving forward. To do so, we established the ‘curriculum committee’ which had representatives from the Ladd Foundation, Impact Society, and the research team. We held a weekly zoom meeting with an agenda to ensure targeted conversation and progress tracking. The ‘creative committee’ also had weekly meetings and then joined the curriculum committee for a biweekly joint meeting to discuss relevant topics.

One of the first necessary decisions for the curriculum committee involved how, and in which order, the program would be structured. A requirement advanced from the Ladd Foundation was that the program be 16 weeks in duration, with a message representing the different ‘Cs’ each week. As can be seen in Table 1, the program was set up to involve (a) a 1-week introductory/engagement phase, (b) a 13-week ‘Cs’ promotion phase, and (c) a 2-week consolidation/maintenance phase. In relation to the introductory phase, this was used to orient the participants to the program by introducing the concept of the ‘Buffalo Mindset’ and having Andrew Ladd create enthusiasm and engagement by describing his experience in sport and highlighting how important it was for him to be a member of a team as a youth. The consolidation phase was used to reiterate important concepts and encourage reflection of the season as well as action planning and forethought for the future.

A second decision involved the 13-week ‘Cs’ promotion phase. As shown in the ‘Sport-Specific 4Cs’ column (Table 1), we determined that competence would receive explicit attention through physical skill-oriented videos for coaches (see description below). In addition, we felt that messaging about competence would be infused throughout all content given the context for the program, whereby ice hockey was seen to inherently involve sport-specific skills and athletic development. In relation to the remaining Cs, we acknowledged that ice hockey represented an interdependent sport and thus, followed team-building intervention literature to leverage the team environment by first introducing connection (e.g., Carron & Spink, 1993). This decision both aligned with the practical metaphor of a buffalo herd envisioned for the program and the academic literature that positions the importance of the social environment for sport experiences (Bruner et al., 2020). It was then decided that confidence and character would follow, respectively, having athletes reflect on themselves and their behaviours.

Another decision pertaining to the Cs required informal literature reviews, conceptual discussions as a research team, and presentations during curriculum team meetings with opportunities for input and feedback. Specifically, the Cs are general overarching concepts (e.g., Côté et al., 2010; Lerner 2006), and so a more granular or nuanced description/ explanation of each was needed to specifically explain ‘what we meant’ by connection, confidence, and character and how they should be translated to the partners and more importantly, to 10-12-year-old youth. As represented in the ‘Foundational Theories/Constructs’ column (Table 1), we identified concepts that would represent an important component of a particular C, while being specific enough to inform story messaging for participants. In doing so, the research team also provided plain language summaries and key takeaway messages to the creative committee to ensure everyone was ‘on the same page.’

Key Considerations. The most important feature of creating an evidence-informed action plan that stays relevant to end-users is ensuring that all partners know what the program aims to achieve and that everyone can seek clarity or raise concerns. For instance, when evaluating a particularly extensive and effective partnership involving the 4-H program in New York State, Agans and colleagues (2020) flagged communication and mutual benefit as critical. For the 1616 Program, it was initially crucial that the researchers clearly distilled knowledge into manageable and user-friendly terminology. This process means that concepts and ideas used in research are translated into a more meaningful and actionable language for program users. In doing so, the frequent meetings and discussions are necessary to ensure that all partners are represented, that consensus is reached, and priorities are established. From a research perspective, it is essential to tailor evidence and adapt views and ideas to the context of the program and the desire of the program users. As researchers, the main questions to be addressed could include: ‘How far can the terminology of established concepts be changed without altering the core features of a theoretical framework?’, ‘Can the program foundation be supported by specific lines of research even though research terminology is not explicitly used?’, and ‘Can program components that have little research evidence still be included, tested, and evaluated?’.

What Happened. Two key actions were taken to ensure the program’s knowledge tools were tailored to the intended program delivery context. First, rather than provide the story-based programming in a setting with a trained facilitator (in the case of Impact Society’s traditional programming, this represents a teacher in a classroom), the 1616 Program was to be remote and self-directed. As such, the Impact Society’s tried-and-tested curricular process of story delivery, guided reflection, and planned action needed to be tailored to remote (i.e., online) delivery.

Second, although decisions for targeting ice hockey were purely preferential and circumstantial based on the Ladd Foundation, relevant implications are worth noting. As Gainforth et al. (2020) described, effectiveness is enhanced when leveraging specific expertise from partners. For example, Andrew Ladd has an extensive network of contacts in the ice hockey community. Whereas these represent significant advantages for recruitment and programming, there is also evidence to suggest opportunities based on ‘sport type’ that can be leveraged. For example, certain sports are predisposed to providing increased opportunities for interactions and identity building compared to others (Evans et al., 2017). Ice hockey is a highly interdependent sport in both task and outcome (Evans et al., 2012), which means interactions and individual tasks and experiences are tied to those of teammates and coaches. Further, sport-specific structural features were considered when creating program tools/products. For example, in addition to interdependence in both practices and games, ice hockey has (a) inherent ‘subgroups’ (e.g., lines, defensive pairings, special teams units) that promote additional opportunities for identity building and the establishment of support and interactions/relations, (b) a ‘shift system’ involving built-in opportunities for reflection and learning (i.e., athletes leave the ice surface to rest and provide opportunities to teammates) and discussions with teammates and coaches, and (c) the necessity of dressing rooms which promote interaction time and increased proximity.

In relation to story/video creation and production, all partners were actively involved. The research team led the development of proposed weekly messages based on the academic literature (Table 1) and provided summary descriptions to the remaining partners. The Ladd Foundation and Impact Society conducted prospective ‘role model’ research and recruitment of elite athletes with potential reflective stories. Once athletes expressed interest, they engaged in a Zoom interview with members of Impact Society and Andrew Ladd to discuss their experiences. Impact Society then proposed a ‘creative brief and story’ that was reviewed by the research team for representation of the topic and alignment with the literature and was then shared with the athlete for feedback/agreement. Once all partners and the athlete agreed, a video shoot was scheduled with the creative committee at the athlete’s convenience. Based on the athlete’s story and intended weekly message, linked reflection questions and ‘live it out’ (LIO) activities (e.g., affirmation statements, journaling, breathing exercises) for the youth, as well as associated support resources for their parents (e.g., daily conversation starter/tips, behaviours to reinforce), and coaches (e.g., daily conversation starter/tips and skills/drills videos), were similarly developed by the curriculum committee and integrated by the creative committee into digital production.

We wanted the programming to influence participant perceptions and experiences, so we used this general objective to define our problem in behavioural terms (Michie et al., 2016). In other words, we proposed activities that could contribute to an enriched sport experience, while also considering where they would occur (e.g., at practice, during the car ride), how often (e.g., weekly), and with whom (e.g., teammates, parents, coaches). From this reasoning, we specifically developed the knowledge products using the COM-B model (Michie et al., 2011). This model suggests that end-users (i.e., youth, parents, coaches) would be more likely to engage with—and benefit from—the program if the activities prioritized capability, opportunity, and motivation. Capability refers to an individual’s capacity to engage in certain behaviours, opportunity reflects an awareness of the social and physical environments’ affordances to engage in the behaviours, and motivation involves the desire and willingness for reflective and autonomic processes (e.g., beliefs, habits). It is important to emphasize that based on Impact Society’s practice and in line with behaviour change theory, we emphasized the need to improve participants’ knowledge (e.g., Do they understand the messages conveyed in the stories and why they are important?), social influence (e.g., Do they understand how their behaviours can influence and be influenced by others?), contextual norms (e.g., Are the concepts introduced compatible with the expectations discussed within their team?), and emotion (e.g., Do the stories and associated activities evoke an emotional response?).

Key Considerations. Creating program content involves a team effort that requires consistent input from all partners. An essential element of this process is having a project manager who coordinates the different tasks, prioritizes strategies, and maintains communication and workflow between partners. Mechanisms to facilitate this process include creating workflow diagrams, establishing meetings, and sharing information and products through documents and e-mails. Researchers must provide regular opportunities to review content while developing tools and products (e.g., videos, websites, etc.). Decisions about entertaining and engaging evidence-informed content is a constant challenge that requires partners to stay open-minded, adaptable, and realistic about what and how practical information can be provided to the users. Indeed, it is important to consider tangible ways to demonstrate power-sharing and collaboration across partners (Jagosh et al., 2015). From a research perspective, questions to help guide the co-production of tools include: ‘How can we best organize workflows across multiple and different groups when designing program content?’, ‘How and when should feedback be provided by different partners when creating a specific tool?’, and ‘How is the integrity of the research process protected in the design and development of entertaining and engaging material?’.

What Happened. Once we had a representative sample of tools/products drafted, a critical next step from the KTA framework was to work with end-users (i.e., athletes, parents, coaches) to understand their perspectives, ensure that the knowledge products developed were relevant and met their needs, and promoted engagement. To do so, three focus groups composed of children (n = 14; 21% female) aged 9-11 years and three of parents/ coaches (n = 14; 57% female) were conducted to garner general impressions and elicit input pertaining to the proposed program content and delivery method. These focus groups also provided the opportunity to trial sample content (i.e., introduction to the program and week 1 videos) and outline program expectations to the target audience (i.e., reflective and action-oriented content). Focus group discussions were selected as they can be conducted remotely and enhance the richness and depth of interview responses by allowing individuals to extend and contrast the opinions of others (e.g., Kitzinger, 2005). For the youth, elements of readability and age appropriateness were explored (e.g., ‘Can you tell me what the reflection/action items are asking you to do?’), in addition to questions addressing what components they enjoyed/did not like (e.g., ‘What was your favourite part about the video?’), what they found useful or not (e.g., ‘Was there anything that you think we should remove?’), and anything they would recommend that we consider including (e.g., ‘If you were to create this kind of program, what else would you do?’). Although the same process was undertaken for parents/coaches, an additional aim was to determine what would make them more or less likely to engage with the program (e.g., ‘If you were going to take part in this program, what are some things that you feel would help to make sure that you watched the video and understood the content clearly?’). The interviews were recorded via Zoom, transcribed verbatim, and analyzed using reflexive thematic analysis (Braun & Clarke, 2021).

As a general summary from the conversations, youth understood the purpose of the reflections and LIO activities, enjoyed professional athletes being involved in the videos, the relatable nature of the stories, the combination of tools/products (i.e., videos, reflections, LIOs), and overall, wanted to sign up for the program. For the parents/coaches, the storylines of the videos were clear, and they were motivated to engage in the daily conversation starter/tips, enjoyed the delivery format, and overall, wanted their child/athletes to participate in the program. They recommended that the content change every week to keep the children engaged and avoid it feeling like ‘homework’ and also cautioned against providing too much information and too frequently to participants. These results were shared with all partners through executive summaries and comprehensive presentations, given their significance for decisions in subsequent iterations of program content creation.

Key Considerations. Getting initial feedback from the program participants is an important initiative to ensure the effective rollout of a program. This step needs to be implemented at an appropriate time during development when tools are refined enough to allow a fair evaluation. Yet, the process should be flexible enough to enable the integration of the participants’ feedback in future iterations of the program tools. Importantly, the researchers must ensure that end-users are not solely used for data collection but have an active voice in the content of the programs (Stoecker, 2009). Researchers are one of the key players in the process of working with end-users and presenting data to partners so that appropriate changes can be discussed and made. As such, the researchers should conduct preliminary analyses to interpret participants’ data that could be conflictual or not representative, so that time and resources are protected. Important questions to consider when working with end-users include: ‘How do partners remain open and flexible to end-users’ input in adapting tools and delivery methods?’, ‘At what stage of knowledge tool development is users’ feedback most productively gathered/ incorporated?’, and ‘How are the feedback results collected, analyzed, and summarized to allow a constructive discussion between all partners?’.

What Happened. Prior to finalizing and implementing the full 1616 Program, based on recommendations put forth in implementation literature, we decided that it should undergo a Proof-of-Concept (PoC) evaluation. Specifically, PoC tests are a recommended evaluation process that enables a preliminary assessment of the impact and effectiveness of a program on a shorter times scale and with an accessible sample (e.g., Thabane et al., 2010). This decision was made for several reasons, spanning timeline, funding, elite athlete availability, but most importantly, the necessity of ‘testing’ feasibility, acceptability, and preliminary effectiveness prior to investing greater time and resources (Czajkowski et al., 2015). Whereas a full description of the PoC implementation and evaluation can be found elsewhere, the basic design was a small-scale roll-out of the program with 11 youth ice hockey teams across North America. The content comprised five representative weeks from the eventual 16-week program, each of which was implemented in its entirety (i.e., beginning with a central story video, followed by reflection items and LIO activities for youth, as well as associated parent and coach resources). As intended for the full program, this pilot implementation was delivered remotely to teams through emails and text messages linked to a central program website.

To assess the pedagogical impact and feasibility/acceptability of the weekly topic structure and associated resources for the program (i.e., the PoC), the research team designed an evaluation plan, in consultation with members of the curriculum and creative committees. The development of this evaluation plan required careful consideration and balance between scientific rigour and pragmatic concerns. For example, as this program development phase strongly emphasized relationship development with the target populations and youth sport organizations, avoiding undue participant burden was critical. Thus, data collection had to be integrated into program implementation to the greatest degree possible. Further, given the extensive time demands on youth sport parents and coaches in general, data collection from participants in these roles needed to be limited to a single time point. To address these (sometimes competing) considerations, a broad and varied evaluation protocol was eventually put forward. This protocol included multiple assessment approaches (e.g., quantitative questionnaires, short open-ended surveys, qualitative interviews/focus groups), each specifically selected to provide maximally rich data in the briefest possible format while spreading the response burden across time and different participant roles. For example, whereas a brief traditional pretest-posttest (i.e., before and after implementation) psychometric questionnaire was trialed for youth athletes, a shorter retrospective pretest/posttest questionnaire (RPP; Little et al., 2020) was used to assess perceptions of change at a single time point each week.

At the end of the PoC testing, evaluation data were first analyzed by the research team and subsequently presented and discussed with both the curriculum and creative committees. This analytic process again required a careful negotiation between the complex analyses often desired by academic audiences (with scientific peer review of the program’s empirical basis being an intended output) and the need for clear and actionable findings to pragmatically inform further program development.

Key Considerations. Flexibility and openness were critical to the process of designing and implementing the PoC evaluation. Although all partners wanted the evaluation to be as scientifically rigorous as possible, the primary purpose of the evaluation was to inform further program development (rather than as a stand-alone research project). In this vein, whereas the research team took the lead on designing the evaluation protocol, continuous open discussion with all partners was critical to making sure relevant data were collected within the existing pragmatic constraints. For example, such discussions allowed the creative committee to advocate for the inclusion of feedback items useful to their unique responsibilities of the program (e.g., visual appeal of program resources). This preparatory dialogue aligned with previous suggestions in youth sport program evaluation, ensuring that all partners were interested and invested in receiving feedback from end-users, rather than the evaluation process being viewed as an external judgement on their work (e.g., Ettekal et al., 2017). As such, prior to designing similar program evaluation protocols, researchers are encouraged to consider: ‘Who needs to use the evaluation data and for what purposes?’, ‘What types of data (e.g., psychometrically validated questionnaires vs. simpler perception or engagement measures, etc.) are needed to most efficiently serve those purposes?’, and ‘How can data analysis and presentation be best tailored to facilitate action towards those purposes?’.

The purpose of this article was to broadly describe our iKT approach to program development and provide a brief overview of the resulting 1616 Program. Figure 2 depicts the ‘participant experience’ for all three targeted end-users: youth, parents, and coaches. In this section, we introduce each engagement product and note relevant aspects that informed their development.

Given that the overarching objective of the 1616 Program was to use elite athletes to introduce important PYD concepts to youth, story creation represented the main portion of our collaborative work. In addition to the main messages conveyed through the stories/videos sent to youth weekly (see Table 1), these were complemented by reflection questions and LIO activities (see Figure 2).

As described previously, through the videos, we sought to build participant knowledge, emphasize the social and normative features of what healthy sport participation should look like, and elicit an emotional response (e.g., excitement, enjoyment).

Youth were also provided with reflection questions to reinforce message consolidation and self-reflection as well as actionable items for them to engage with throughout the week. The reflection questions were meant to help with understanding the key message (i.e., Capability) and to have them consider the applicability of the lesson in their own lives and how it might be impactful for them (i.e., Motivation). The LIO activities represented a way for youth to consolidate the messages and actively apply this new knowledge into their own lives (i.e., Opportunity). For instance, athletes were encouraged to write ‘we go into the storm together’ onto the shaft of their hockey stick to promote the buffalo mindset in real time.

We aimed to provide useful tools/products for both parents and coaches so they could better support their children/athletes and help reinforce the program’s messages, while also ensuring that time demands were not a barrier for use (Dorsch et al., 2019). Every week, parents were sent a brief summary of the discussed topic and asked to watch the video sent to their children. In addition, they were provided useful ‘tips’ to inform their behaviours during the car ride to and from the arena and also general conversation starter suggestions to reiterate the concept of the week based on the academic literature (Tamminen et al., 2017; Tamminen et al., 2020). Finally, they were provided with useful examples of behaviours that they could ‘catch their kids’ doing to provide positive affirmation to their child’s LIO behaviours.

Coaches were provided with a summary of the weekly topic and practical ‘tips’ for reinforcing it based on transformational coaching behaviours (e.g., Turnnidge & Côté, 2017) and team-building literature (e.g., Paradis & Martin, 2012). In addition, because this is an ice hockey program, the coaches received a video each week from Andrew Ladd and Adam Oates with a specific hockey skill that they could work on with their athletes. These videos were a tool used to improve engagement from coaches but also to emphasize the competence ‘C’ from PYD.

While acknowledging that the iKT process involved in developing the 1616 Program was in some ways unique, especially with respect to the professional video/digital media production support, fundraising capacity, and relevant networks of the Ladd Foundation, we feel that generalizations can be made to other iKT efforts involving multiple partners and complex decision-making environments. In reflecting on the ‘lessons learned’ from this iKT program development process, we found significant alignment with the guiding iKT principles proposed by Gainforth and colleagues (2020), and recommendations from the research-practice partnership literature. Accordingly, we felt that several advanced principles and recommendations resonated with our experience developing the 1616 program and warrant further discussion.

First, we emphasize the importance of establishing relationships based on trust and respect. None of the program development partners had previously worked together and were all coming from different disciplines with varying assumptions and modes of working. Therefore, getting to know one another and understanding the unique expertise each brought to their role was critical for effective functioning and sustainability in a complex, fast-paced environment (e.g., Jagosh et al., 2015). Second, it was necessary to prioritize open, honest, and responsive communication. While also facilitative of the trust and respect noted first, intentionally creating the space (and indeed, expectation) for all partners to communicate their perspectives and consider alternative views allowed all voices to be heard and potential challenges to be identified and addressed pre-emptively rather than reactively (e.g., Agans et al., 2020). Third, we recognized that every partner brought specific and unique expertise, and thus, the need to share in decision-making responsibilities was at the forefront. The multifaceted nature of the 1616 Program meant that no one partner had all the necessary tools to tackle any element on their own; this quickly-realized reliance on each other’s expertise meant that everyone was committed to shared decision-making and thus helped to ensure that all were ‘pulling in the same direction’ (e.g., Melton et al., 2022). Fourth, we prioritized the need to ensure that all partners were flexible and receptive to tailoring the research, production, and delivery components of the program based on pre-defined aims, the context, and practical and financial constraints. Had any of the partners demanded strict adherence to their (or their discipline’s) singular traditions, progress on program development would not have been possible. The breadth and scope of the intended program necessitated the negotiation and integration of the needs of all invested partners. This flexibility was also greatly facilitated by the trust, respect, and open communication noted above (e.g., Ettekal et al., 2017). Finally, we made sure that all partners benefitted from their involvement. Given the significant time and resource investment required of all partners over a sustained period, the leadership of the Ladd Foundation was vital to ensuring that each of the partners found meaning in their participation. Through open discussions and regular check-ins, explicit attention was paid not only to the progress of the program’s development but also to the quality (and therefore reward) of the collaborative process.

In terms of next steps, the iterative and collaborative nature of our partnership continues. The PoC evaluation has provided extensive feedback for all partners to consider, both in relation to the content and delivery of the program, but also for future large-scale evaluation protocols. As several examples, we believe that greater inclusion of end-users (youth, parents, and coaches) during content creation and refinement would further strengthen our iKT approach, and so a sample of ‘super-users’ willing to review and discuss iterations of knowledge tools will be recruited and more formally involved. Similarly, given the magnitude of data collected during the PoC and the scope expected for full-program implementation, we plan to use the RE-AIM framework (Glasgow et al., 1999) to best evaluate the implementation and impact of the 1616 Program, which also aligns with recent sport intervention reviews that emphasize the need for transparency and comprehensive assessments in this context (e.g., Bruner et al., 2021; Whiltey et al., 2019).

Source of Funding: This project was supported by the Mitacs Accelerate Program (Application Ref. IT26518) to support two graduate students/internships.