Article Text
PDF
Abstract
Warming-up Hockey (WUP) is an effective injury prevention programme to reduce acute field hockey injuries among youth. This paper describes the process evaluation of the nationwide scaling-up. We conducted a mixed-methods process evaluation from September 2019 to December 2020 according to the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework, focusing on the intervention and its implementation. We collected data through questionnaires, interviews and web/app analytics. Participants were trainers/coaches, technical/board members of hockey clubs (TBMs) and employees of the Royal Dutch Hockey Federation (KNHB). In total, 226 trainers/coaches (61 via WUP and 165 via training courses) and 14 TBMs filled in questionnaires. Ten individuals (four trainers/coaches, four TBMs and two KNHB employees) participated in semistructured interviews. The study showed the following results according to the RE-AIM framework. Reach: According to web/app analytics, 1492 new accounts were registered. Effectiveness: Overall, users were satisfied with WUP and the implementation strategies, and believed WUP could reduce field hockey injuries. Adoption: 63% of the trainers/coaches (enrolled via WUP) indicated they used WUP. Implementation: Most trainers/coaches did not use WUP during every training session or match. Most TBMs promoted WUP in their club. Implementation barriers included lack of integration with other training programmes, ‘know-it-all’ trainers, lack of supervision on WUP use and delayed start of implementation. Facilitators included perceived added value, information need on injury prevention in small clubs and tailored communication. Maintenance: Users planned to use WUP occasionally. The KNHB intended to integrate WUP in their newly developed Knowledge Platform. To conclude, WUP was evaluated as a useful programme, but adherence to WUP was challenging. Timely preparation and creating an implementation plan based on stakeholder input, including communication at key moments during the sports season and tailored communication, were found to be important during implementation. Findings can be useful for others planning to implement evidence-based injury prevention programmes on a larger scale.
- Sporting injuries
- Implementation
- Intervention
- Field hockey
- Prevention
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Participation in field hockey includes a risk of injury. Injuries can be reduced by using evidence-based injury prevention programmes. However, implementing these programmes in practice remains a challenge. Multiple factors operating at many levels play a role during implementation, and research concerning effective implementation processes of injury prevention programmes in real-word contexts is desirable.
WHAT THIS STUDY ADDS
This study contributes to narrowing the science practice gap by systematically evaluating the nationwide scaling-up of the injury prevention programme Warming-up Hockey (WUP) and its implementation in accordance with the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The study demonstrated that WUP is a useful programme, but in practice, adherence to the programme was a challenge. Sports federations can possibly stimulate adherence to injury-preventive interventions by, for instance, integrating stand-alone interventions in broader programmes, greatly emphasising its added value among all target groups and stimulating clubs’ supervision of intervention use.
Also, results of the study can be beneficial to other sports federations and researchers who would like to implement injury prevention programmes on a larger scale. This evaluation shows the importance of timely preparation and creating an implementation plan based on stakeholder input, including communication at key moments during the sports season (ie, before and at start of the season and just before the start of the spring competition) and tailored communication to target groups. Since nationally implementing an intervention takes time, a long-term implementation effort is needed.
Introduction
Regular physical activity carried out through participation in sports has many positive health effects for children and adolescents1 2 but also includes risks for injury. Sports injuries can lead to individual players’ health burden, extensive lay-off periods, and direct and indirect costs for society.3 Field hockey is a popular team sport in the Netherlands,4 however, around 240 000 field hockey injuries were registered in 2019,5 most affecting the lower limbs.6
In order to prevent injuries, many evidence-based injury prevention programmes have been developed7 and evaluated, often consisting of injury preventive exercises.8–12 To have a positive impact on injury prevention, effective evidence-based programmes need to be widely adopted, implemented and maintained.13 Translating evidence-based sports injury prevention programmes to the real-world context can be challenging due to multiple factors, for example, social influence and broader cultural norms,14 which can interact at many levels (eg, individual athlete, trainer/coach, sports club and the sports federation).15 The Translating Research into Injury Prevention Practice framework (TRIPP) acknowledges the importance of understanding this real-world implementation context (step 5) and of the evaluation of implementation in these contexts (step 6).16 In this way, it complements the sequence of prevention model about intervention development (steps 1–4).17 18 By using the ‘TRIPP framework’, implementation of injury prevention programmes in real-world sports settings can be optimised.12 In order to move the field forward, clearly and properly reported findings on implementation processes of proven effective interventions are desirable, to know what (does not) work(s) and to replicate these effects in the implementation of similar future interventions.19 20
In order to gain knowledge on implementation processes in a real-world context, we evaluated the natural course of nationwide injury prevention programme uptake in the Netherlands. This study describes the process evaluation of the scale-up of Warming-up Hockey (WUP) in the Netherlands. More specifically, WUP and its accompanying implementation strategies were evaluated using the framework for evaluating implementation research: the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework.
Methods
We conducted a mixed-methods process evaluation.
The intervention programme and its implementation strategies
To reduce injuries in field hockey, the Royal Dutch Hockey Federation (KNHB) and the Dutch Consumer Safety Institute (VeiligheidNL), together with national field hockey and injury prevention experts, developed an injury prevention programme for trainers/coaches of youth field hockey teams: WUP.21 WUP was positively evaluated on relevancy, satisfaction and usability,21 and proved to be effective in reducing acute field hockey injuries and injury burden.10
WUP, available as an app and website, consists of sex-specific and age-specific structured and evidence-informed exercises with an explanation via text and video. Exercises should be executed before training and match sessions in youth field hockey (see online supplemental appendix I for screenshots WUP) and consist of a preparation phase (ie, agility and cardiovascular warm-up exercises), movement skills (ie, stability and flexibility exercises) and field hockey skills (ie, speed and strength exercises in field hockey situations). After a trainer/coach logs in and signs up a team, a tailored training programme of 40 weeks is created covering the preseason, competition season and postseason.
Supplemental material
To implement WUP nationally, an implementation plan was drawn up22 by an implementation expert with input from relevant stakeholders (see table 1). Based on this implementation plan, the KNHB promoted WUP on a regular basis between September 2019 and December 2020 (see online supplemental appendix II) among trainers/coaches and technical/board members (TBMs) by using several implementation materials, like articles/messages and videos in digital newsletters, on the website of the KNHB or hockey.nl, on social media (Facebook, Twitter and Instagram) and, for example, a PowerPoint presentation in webinars, training courses and masterclasses. Also, a toolkit with communication materials was promoted among TBMs to support them in implementing WUP in their clubs. Online supplemental appendix III provides an overview of this implementation process.
Overview of implementation goals and strategies*
Participants
Participants were trainers/coaches aged 16 years and older of youth field hockey teams in the Netherlands and TBMs, who registered for WUP from September 2019 to October 2020 (ie, WUP users). In addition, trainers/coaches who took part in KNHB training courses in this time period were invited to participate in the study. Trainers can be defined as training the team one or multiple times per week (ie, teaching hockey skills) and coaches as coaching the team during matches (ie, deciding on the line-up and providing tactical instructions). Furthermore, KNHB employees who were involved in executing the implementation plan were included in the study.
The RE-AIM framework
The process evaluation was performed according to the RE-AIM framework,23 which can be used to evaluate the introduction of intervention strategies in a practical (sports) context.24 Table 2 describes how we operationalised the RE-AIM components.
Description of the operationalisation and measurement of the five components of the RE-AIM framework
Data collection
We collected data through questionnaires, semistructured interviews and WUP/Google analytics between September 2019 and December 2020 (see table 2). Online supplemental appendix IV provides a detailed overview of methods used and study participants.
In questionnaires, developed by sport injury prevention experts of the Dutch Consumer Safety Institute and an implementation expert of the Amsterdam UMC in collaboration with a KNHB employee, participants were asked demographical questions and questions concerning WUP related to the RE-AIM framework (see online supplemental appendix V). All WUP users who checked the opt-in box for participation in research during registration for WUP from September 2019 to March 2020 received a baseline (T0) questionnaire. There were two types of questionnaires, one for trainers/coaches and one for TBMs. The number of follow-up questionnaires ranged from one to three (T1/T2/T3), depending on the moment of enrolment in the study. Also, WUP users who did not respond to T0 or registered for WUP from April to October 2020 and checked the opt-in box for participation in research (hereafter referred to as ‘late entrants’) were contacted (again) to complete a questionnaire. The questionnaire for late entrants was a mix between the baseline questionnaire and a follow-up questionnaire. Trainers/coaches from the KNHB training course received only one separate questionnaire. All participants provided informed consent through the online questionnaire.
In the questionnaires, participants could express interest in participation in an interview: convenience sampling was used. KNHB employees were asked through email for their participation in interviews. Interviews were held (mostly by phone) to gather a deeper understanding of the implementation of WUP (see online supplemental appendix VI for the topic guide). They were conducted by MHC and AB at different time points, until data saturation was reached .
Online analytics (WUP/Google analytics) were used to determine the number of trainers/coaches reached (see for operationalisation table 2: Reach of WUP) and which implementation strategies they engaged with (see for operationalisation table 2: Adoption of implementation strategies).
Data analysis
Data originating from the questionnaires were analysed using SPSS (V.25). Descriptive statistics were used. We included all fully or partly completed questionnaires in the data analysis. If the same participants completed the same questions at different time points, answers were aggregated or averaged (see online supplemental appendix V). For numeric scores this was regarded as valid, given the lack of variation within individuals. Of the statements in the questionnaire asked on a 7-point Likert scale (disagree-agree), answers were categorised: 1–3 disagree, 4 neutral, 5–7 agree.
A thematic analysis was performed on the qualitative data.25 First, transcribed interviews and field notes were read to gain familiarity with the data (compiling). Second, AB and FvN open-coded four interviews using initial codes, which were subsequently compared in a team meeting and clustered into overarching codes in compliance with table 2 RE-AIM framework operationalisation, and recorded in a final codebook (see online supplemental appendix VII). Using the generated codebook, AB independently coded the remaining interviews (disassembling) along with corresponding quotations. By analysing existing codes, underlying themes were identified (reassembling). Quotes were selected to illustrate quantitative results.
All data were treated confidentially. Data were securely stored and could only be accessed by MHC and AB.
When presenting results from trainers/coaches, trainer/coach refers to trainers/coaches who participated in the study via WUP. When presenting results from trainers/coaches who participated in the study via the KNHB training course, this is mentioned specifically.
Results
Until October 2020, when the last questionnaire was sent, the total number of WUP users who checked the opt-in box for participation in research was 298 (24% of all registered accounts). Of these 298, 75 WUP users (25%) participated in the study: 61 trainers/coaches (14 enrolled in the baseline questionnaire and 47 in the late entrants’ questionnaires) and 14 TBMs (2 enrolled in the baseline questionnaire and 12 in the late entrants’ questionnaires). Of the trainers/coaches from the KNHB training course, 165 out of 650 (25%) enrolled in the questionnaire (see figure 1).
Flow chart of participants. *Number differs from total number of registered accounts (1492) in the implementation period, since the implementation period lasted longer than the research period (last questionnaire was sent in October 2020). KNHB, Royal Dutch Hockey Federation; WUP, Warming-up hockey.
Concerning interviews, data saturation was reached after 10 interviews with different stakeholders: 4 trainers/coaches, 4 TBMs and 2 KNHB employees (see table 3).
Participant characteristics
Reach
Warming-up Hockey
From September 2019 to December 2020, the total number of new WUP accounts was 1492 (7% of the estimated population of trainers/coaches). The website generated 12 000 sessions from 9400 users (unique visitors). Also, 6000 users (unique visitors) visited the WUP app (see online supplemental appendix VIII).
Effectiveness
Warming-up Hockey
On overage, satisfaction by trainers/coaches (N=16) who reported using WUP was rated a 7.2 (SD=1.8, min: 1, max: 9) on a scale from 1 (completely not satisfied) to 10 (very satisfied): ‘WUP gave practical input for a warm-up. Exercises were good’” (trainer/coach). Among TBMs (N=7) WUP was averagely rated a 7.6 (SD=0.5, min: 7, max: 8). Of the trainers/coaches participating in the study via the KNHB training course, 74% (N=17) were satisfied with WUP. Not all users were equally satisfied, some missed information about the goal of the exercises: ‘I miss the link between exercises and injury prevention. Why do we use these exercises? Which muscle groups do we train and how are injuries prevented?’.
Trainers/coaches (N=16) who used WUP rated its user-friendliness a 7.0 (SD=2.3, min: 1, max: 10) and TBMs (N=7) a 7.3 (SD=0.8, min: 6, max: 8) on a scale from 1 (completely not user-friendly) to 10 (very user-friendly). Users indicated there were some temporary technical problems, such as logging in: ‘I had to keep logging in, and it was problematic to select and add teams.’ (trainer/coach). However, overall, WUP was experienced as user-friendly: ‘WUP works fine. I can find everything easily and don’t have to look for hours.’ (trainer/coach). Especially videos were well received by the trainers/coaches: ‘I like the videos in WUP. First, I read the text, and then I can easily check if I understood the exercise by watching the video.’
Concerning perception of impact, 85% (N=11) of the trainer/coaches and 92% (N=12) of the TBMs believed WUP could reduce injuries among field hockey players: ‘My team consists of really young players, who generally have fewer injuries. However, I think performing a good warm-up is important in order to prevent injuries.’ (trainer/coach).
Implementation strategies
Trainers/coaches and TBMs indicated their overall satisfaction with the implementation strategies: 82% (N=29) of the trainers/coaches and 73% (N=8) of the TBMs agreed to be satisfied. More specifically, 89% (N=24) of the trainers/coaches and 70% (N=7) of the TBMs agreed the message was clear, 81% (N=22) of the trainers/coaches and 70% (N=7) of the TBMs agreed they felt addressed by/could relate to the message, 85% (N=23) of the trainers/coaches and 80% (N=8) of the TBMs agreed the utility of WUP was clear in the message.
Trainers/coaches participating in the study via the KNHB training course (N=30) rated their satisfaction with the implementation materials a 7.1 (SD=1.9, min: 3, max: 10) on a scale of 1 (not satisfied) to 10 (very satisfied), the clarity of the message a 7.1 (SD=1.8, min: 3, max: 10), feeling addressed by/relating to the message a 7.1 (SD=2.2, min: 2, max: 10) and the clarity of the utility of WUP in the messages a 7.5 (SD=1.8, min: 2, max: 10). As one trainer/coach stated: ‘The message was very clear. For every trainer who uses common sense, the information was sufficient.’
Adoption
Warming-up Hockey
Of the trainers/coaches participating in the study via WUP, 63% (N=22) indicated they used WUP. Of the trainers/coaches participating in the study via the KNHB training course, 29% (N=23) had used WUP. The non-users mentioned different reasons for not using WUP, arguing for example that available content did not match (exercises already being familiar—no new information—and the desire for more dynamic exercises) or WUP was not sufficiently user-friendly (especially technical defects when logging in).
The most common reason trainers/coaches gave for using WUP was inspirational (75%, N=24), followed by reducing injuries (40%, N=13): ‘I use WUP for inspiration only. I just look what exercises WUP suggests and adapt them to my team.’ (trainer/coach). Of the trainers/coaches using WUP, 88% (N=28) agreed they could use WUP independently.
Barriers and facilitators for adopting WUP
Regarding the programme itself, a hindering factor concerning adopting WUP was the lack of integration with other programmes: ‘I think Warming-up is too specific to be a stand-alone app. I don’t like having a different app for every aspect of field hockey.’ (trainer/coach). A facilitating factor was the perceived added value of WUP. Satisfaction with WUP was ambiguous: trainers/coaches were satisfied with the content (variety of exercises), but user-friendliness could be improved.
Regarding characteristics of trainers/coaches, adoption by the trainer/coach was hindered by, among others, ‘know-it-all’ trainers: ‘A hindering factor for using WUP is the stubbornness of trainers. They think they know it all.’ (TBM). Furthermore, the amount of experience of trainers/coaches both facilitated and hindered WUP use. More experienced trainers would be less likely to use WUP due to existing knowledge of how to structure a warm-up/training, but are also more likely to use WUP due to an increased interest in injury prevention. Also, it was mentioned that trainers are often volunteers who do not have time to learn something new, but on the other hand, they sometimes struggle to structure a training. WUP can be of help: ‘Less experienced trainers, often volunteers, could use it, since it is a tailor-made programme.’ (trainer/coach).
Additionally, COVID-19 hindered WUP usage to some extent: ‘Due to corona we could not enter the field before the training or match. We had to execute the warm-up faster than we normally do.’ (trainer/coach). For a complete overview of the barriers and facilitators, see table 4 (for barriers and facilitators related to ‘Organisation’ and ‘Context’, see the ‘Implementation’ section).
Barriers and facilitators during WUP implementation
Implementation strategies
The KNHB applied several implementation strategies to disseminate WUP among the target population, for example, publishing articles on their websites (www.knhb.nl and www.hockey.nl), sending newsletters to 945 clubs (overall, 50% opened them, and 4% clicked on the WUP link), and organising webinars and masterclasses (165–204 participants). Also, the KNHB posted messages on social media, among others, videos of WUP (between 287 (Twitter) and 9913 (Instagram) views each time). The implementation strategies resulted in a peak of users in October 2019 (after the start of implementation) and February/March (start of spring competition) and September 2020 (start of hockey season) (online supplemental appendix VIII).
Data from questionnaires showed most WUP users were familiar with WUP via the KNHB (trainers/coaches: 51%, N=25, TBMs: 77%, N=10) and through the website (trainers/coaches: 60%, N=15, TBMs: 50%, N=5), followed by social media (trainers/coaches: 28%, N=7, TBMs: 50%, N=5). Some trainers/coaches also heard about WUP through their club (22%, N=11), mostly via the board (55%, N=6). Implementation materials typically noticed were articles/messages (trainers/coaches: 31%, N=12, TBMs: 42%, N=5). Of the trainers/coaches participating via the KNHB training course, 29% (N=40) heard about WUP in their training course (did not hear: 46%, N=63, don’t know: 25%, N=34). These trainers/coaches knew about WUP mostly through text on a PowerPoint slide (88%, N=30) and/or an oral explanation by their teacher (74%, N=25).
Implementation
Warming-up Hockey
Trainers/coaches indicated how frequently they used WUP: of the trainers, 50% (N=9) used WUP repeatedly (in some or (almost) all training sessions), and 50% (N=9) used WUP rarely (in a few training sessions or none). Most of the trainers used the training exercises separately (60%, N=9), while 40% (N=6) used a team-specific training scheme. Of the coaches, 50% (N=11) used WUP repeatedly in matches (in some or (almost) all matches), and 50% (N=11) used WUP rarely in matches (in a few matches or none). Most coaches used the training exercises separately (60%, N=9), 53% (N=8) used the general match warm-up, and 27% (N=4) the specific match warm-up exercises for players and keepers. Online web analytics supported data from the questionnaires that training exercises were mostly used separately (web page views: all exercises 3598, team-tailored exercises 1328).
When focusing on exercise execution, interviews revealed that exercise instructions were not strictly followed by trainers/coaches: ‘I use different apps to prepare a training. I like doing it that way. Then, I can adapt the warm-up to the training and the weather conditions.’
Implementation strategies
Implementation strategies provided TBMs tools to promote WUP within their clubs. Results of the late entrants’ questionnaires showed that all TBMs promoted WUP, 75% (N=7) among trainers, 50% among coaches (N=5) and 50% (N=5) among other TBMs.
Barriers and facilitators in implementing WUP
Technical/board members
Interviews showed that various organisational characteristics of clubs played a role in the implementation of WUP (see table 4). Barriers mentioned were the lack of possibilities for supervision on WUP use, staff turnover, and a lack of time and capacity within small clubs to implement WUP: ‘Bigger clubs have the opportunity to recruit employees to manage the implementation of WUP. In smaller clubs, a few people do everything.’ (TBM). On the contrary, quite some informal communication takes place within small clubs, which can facilitate the implementation of WUP. However, it can also cause unclarity, since not all agreements are formalised. Another facilitating factor within small clubs was the lack of knowledge about injury prevention: ‘Small clubs will especially benefit from WUP, since they have less knowledge about injury prevention than bigger clubs.’ (TBM). Furthermore, in all clubs, the enthusiasm of TBMs was regarded as a facilitating factor.
Context: KNHB
Barriers and facilitators have influenced the execution of implementation strategies by the KNHB (see table 4).
First, the KNHB positively evaluated the structural preparation of the implementation (eg, developing an implementation plan). However, the timing of the preparation phase (during the summer holiday) was impractical due to absent employees. Also, some technical difficulties were causing delays in launching the updated version of WUP. That is why WUP could not be used from the start of the field hockey season (September 2019), but from October 2019 onwards. As a consequence, communication with trainers/coaches was delayed. Trainers/coaches were hindered from integrating WUP in their routine from the start of the season: ‘The KNHB should reach trainers at least 1 or 2 weeks before the start of the field hockey season, so they can integrate it in their plans.’ (TBM).
During the implementation phase, prioritising communication about WUP proved challenging due to other relevant topics (eg, information about rules, competition) and other, more appealing topics, such as improving hockey skills. Also, communication about consequences of the COVID-19 for field hockey was temporarily dominant from March 2020. Around that time, some implementation strategies were not executed due to COVID-19 (see table 1). However, later on, the KNHB increased its promotion of WUP again, for example, boosting online implementation strategies. The COVID-19 situation offered possibilities to communicate about WUP, since injuries were more likely to occur after a period of non-activity: ‘When corona restrictions were eased, it was good timing for communicating about performing a warm-up.’ (KNHB).
Being able to send tailored messages to different target groups was also a facilitator during implementation. Although reaching TBMs remained difficult. Digital KNHB newsletters, for instance, did not reach relevant club members (eg, only the secretary): ‘The club receives a digital KNHB newsletter and usually sends it to all board members, but the newsletter is not always read, and there is a big gap between sending and reading.’ (TBM).
Furthermore, the KNHB could only stimulate WUP use through communication, since trainers/coaches and clubs decide on WUP usage themselves. In this respect, the non-binding role of the KNHB was a barrier: ‘The KNHB cannot make WUP use obligatory. I think the main goal of the KNHB is to facilitate field hockey in all its aspects.’ (KNHB).
Maintenance
Warming-up Hockey
Of the trainers/coaches that completed baseline or late entrant questionnaires in the first half of the hockey season (2019–2020 or 2020–2021), 76% (N=22) agreed that they intended to use WUP throughout the season, and 55% (N=16) agreed they intended to use WUP in every training/match.
Implementation strategies
Technical/board member
Most TBMs indicated they were planning to promote WUP within their club: ‘I plan to promote WUP through our ‘hockey school’, since WUP is a useful platform. Most trainers/coaches don’t know about it yet.’ (TBM).
Royal Dutch Hockey Federation
The KNHB intended to integrate WUP in their Knowledge Platform, entailing information about match rules, education, training, coaching, etc. WUP could be part of ‘training and coaching’, in which exercises and training schemes are offered (launch expected after the implementation study): ‘The biggest impact we can make is by integrating WUP in the ‘Knowledge Platform’ (KNHB). Also, the KNHB planned to continue promoting WUP through social media and the website. Furthermore, WUP will be integrated into education offered to trainers/coaches and TBMs.
Discussion
Clearly and properly reported findings concerning the implementation of effective sports injury intervention programmes are scarce.19 20 This study contributes to narrowing this science practice gap. Questionnaires, interviews and online analytics generated valuable information on how implementation strategies implementing WUP were deployed and how WUP was evaluated. The implementation of WUP did not exactly go as planned due to a delayed start of the implementation and the COVID-19 outbreak. Implementation strategies were effective in attracting new WUP users, although should be refined to stimulate adherence. Future integration of WUP in existing KNHB platforms might help to stimulate adherence. Nevertheless, WUP users were satisfied with the intervention programme.
WUP evaluation and its implementation in compliance with RE-AIM
Regarding reach, an estimation could be provided of the percentage of trainers/coaches who registered for WUP, namely 7%. During the study period, reach may be influenced by the COVID-19 pandemic, as playing field hockey was temporarily not possible—which could have made WUP less relevant. All in all, a start with implementing WUP has been made, but there is more ground to cover. As long as WUP is a stand-alone programme, reach can probably be increased by continued use of the implementation plan, including the unused implementation channels/activities (eg, using role models). Especially social media channels seemed to work, so there might lie opportunities there (eg, working with influencers). However, it needs to be studied to what extent these channels can reach the whole target group and what the exact message should be (to stimulate adherence).
In this study, effectiveness was measured as the satisfaction with WUP, user-friendliness, and the perception of impact, since the effect of WUP on injuries was already studied.10 Our results showed that most trainers/coaches and TBMs were satisfied (overall and concerning user-friendliness) with WUP, many of them perceiving WUP as injury-preventive. Satisfaction with the delivered programme,26 perceived ease of use, and perceived usefulness can contribute to positive attitudes towards WUP and the intention to use or the actual use of WUP.27 28 Also, an implementation study regarding an effective app to prevent ankle sprains acknowledged a positive user-experience can contribute to (structural) use of an intervention,29 although it does not guarantee this. Concerning satisfaction with implementation materials, most WUP users were satisfied. Probably, thorough preparation of the implementation phase contributed to this, for example, organising implementation sessions with the target groups, drafting an implementation plan, and relying on the expertise of the KNHB in reaching the target groups.
Not everyone who registered in WUP adopted WUP and implemented it as intended. Trainers/coaches used WUP mainly inspirationally and did not use it during every training/match. Lack of adherence (the degree to which an individual chooses to pursue the suggested behavior30) was also a barrier in other injury prevention implementation studies.31 Adherence is a known challenge in implementing sports injury prevention intervention programmes, since it is a complex process in which different factors play a role.32 Concerning WUP, it can be influenced by, for instance, characteristics of the intervention itself, of the trainer/coaches or actions taken by the TBMs. In a broader context, the KNHB played a role in adherence by stimulating the target groups to use WUP through online implementation strategies at key moments during the sports season, for example, just before the start of the spring competition. However, these strategies resulted in new registrations, but affected adherence to a lesser extent. Perhaps KNHB strategies should focus more on influencing clubs and trainers/coaches’ perception towards injury risk and prevention first, possibly eventually resulting in greater programme adoption and adherence.33 When looking at how exercises were used, it turned out that instructions of the WUP exercises were not strictly followed. This is probably linked with trainers/coaches using WUP mainly for inspirational purposes. Not using an injury prevention programme as intended can reduce the programme’s effectiveness34—in literature, referred to as the ‘voltage drop’.35 36 However, adapting an injury prevention programme can also positively affect its effectiveness, as it might be better tailored to the specific user or context. Therefore, it can be argued that this is more desirable than urging users to use the injury prevention programme exactly as intended, which might cause them to stop using it completely. This is worthy of future research.
Concerning ‘maintenance’, on the individual level, most of the trainers/coaches intended to use WUP until the end of the field hockey season. However, fewer trainers intended to use WUP in every training/match. They planned to keep using it mainly inspirationally. It is unknown if trainers/coaches’ use did continue, since WUP use, for example, 6 months after the implementation period, was not studied. There are usually several challenges in long-term implementation at the setting level, such as staff turnover, slackening attention, a lack of clarity regarding goals, vision and strategy, and a lack of sense of responsibility.37 38 By embedding WUP in their Knowledge Platform and integrating WUP in education for trainers/coaches and TBMs, the KNHB partially overcomes these possible pitfalls. However, due to the non-binding character of WUP, active promotion among TBMs and trainers/coaches likely remains necessary to stimulate structural use.
Strengths and limitations
A strength of the study is the evaluation of the natural course of implementing WUP, providing insight into the real-world implementation process. Furthermore, we used the well-known RE-AIM framework to guide the evaluation.19 Although, it originally did not assess facilitators and barriers. Therefore, we complemented RE-AIM by qualitative assessments. This can further strengthen RE-AIM by providing more contextual information about the implementation process.39 Moreover, all WUP users were invited to participate in the study and share their experiences, also if they were negative. In this way, we avoided recruiting only enthusiastic participants, although still participants with an interest in the topic might be included. In addition, by conducting interviews, we were able to capture a good overview of WUP use in practice. Lastly, by combining both quantitative and qualitative data, we were able to capture the ‘story’ behind the data.
A limitation of the study could be the relatively low number of respondents. It turned out difficult to include participants in the questionnaire. One should take this into account when interpreting the results. Yet, we feel that we were able to gather as much information as possible concerning the implementation of WUP. The mixed-methods nature of the study contributed to this. Nevertheless, we only evaluated the implementation of WUP for 18 months, partly during the COVID-19 pandemic. It would be of added value to study the implementation for an additional year, since the implementation of an intervention programme takes time.40
Conclusion
In this study, WUP and its nationwide scaling-up were evaluated among trainers/coaches, TBMs and KNHB employees using the RE-AIM framework, including assessing barriers and facilitators for implementation. The study showed that WUP is believed to be a useful programme, but adherence was a challenge. Steps should be taken to increase adherence among trainers/coaches, such as integrating WUP into a broader programme. Timely preparation and creating a implementation plan based on stakeholder input, including communication at key moments during the sports season and tailored communication, were found to be important during implementation. Since only a small part of the trainers/coaches registered for WUP in 2 years, in which field hockey activities were partly hindered through COVID-19, implementation activities should continue. Implementation experiences and barriers and facilitators for using and implementing WUP identified in this study can be beneficial to other sports federations and researchers who plan to implement intervention programmes.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
The study was assessed by the Medical Ethics Committee (METc) of Amsterdam University Medical Centers—VU University Medical Center location (2019.456), declaring that the protocol did not fall under the scope of the Medical Research Involving Human Subjects Act (Dutch law). The study was conducted in compliance with the Declaration of Helsinki.
Acknowledgments
The authors would like to sincerely thank the Royal Dutch Hockey Federation for their contribution in this project. The authors would also like to thank all participants of the questionnaires and interviews.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @ellenkemler, @FemkeVanNassau
Contributors MHC contributed to the design, acquisition of data and analysis, and wrote the manuscript. EK contributed to the concept, design and provided revisions to the manuscript. AB contributed to the design, acquisition of data and analysis and provided revisions to the manuscript. FvN contributed to the design, analysis and also provided revisions to the manuscript. MHC is the guarantor.
Funding This study was partly funded by ZonMw (536001008), the Netherlands Organisation for Health Research and Development.
Competing interests One author (FvN) is an editorial board member.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.