Article Text
PDF
Abstract
Objectives A massive open online course (MOOC) has the potential to help address the public health burden of concussion across all levels of sport and leisure activities. The main objectives of this study were to document the volume of participation and to estimate the impact of a MOOC on concussion protocol implementation.
Methods Between April 2016 and October 2018, four editions of a French-language MOOC on concussion were presented. Each of the six modules contains a section presenting the main learning content and a section proposing a reflective process to support the implementation of a concussion protocol in the participant’s environment. The proportion of registrants who achieved successful completion of the course was the main outcome. Surveys were also used to document the types of participants and their intent to implement or update a protocol.
Results Thirty per cent of 8368 registrants successfully completed the course. Of the 3061 participants who completed a survey about their background, 58.8% were healthcare professionals, 16.3% were sport or school stakeholders, and 10.1% were parents or persons who sustained a concussion. Of the 1471 participants who completed a survey about their intent to implement or update a concussion protocol in their environment, 39.4% answered positively.
Conclusion This study describes the first use of a MOOC to address the issue of concussion. The experience of a French-language MOOC shows promising results supporting the use of this innovative educational strategy as part of the solution to the public health issue of concussion.
- concussion
- education
- public health
- sports & exercise medicine
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What are the new findings?
The completion rate documented suggests that a massive open online course (MOOC), as a knowledge transfer modality, can help to address the public health burden of concussion.
The high satisfaction rate suggests that the educational strategies used within this MOOC were relevant to the participants.
A MOOC on concussion can virtually bring together stakeholders from a broad range of perspectives.
How might it impact on clinical practice in the near future?
By increasing public awareness and supporting sport and school settings in the implementation of proper concussion prevention and management, a MOOC in concussion can reduce the healthcare burden of concussion and facilitate the proper implementation of the clinical counselling provided following this injury.
Introduction
Concussion as a public health issue
Media attention regarding the catastrophic complications of sport-related concussion (SRC), such as the injury suffered by Zackery Lystedt in 2006,1 and recurrent concussion in high-profile athletes contributed to the identification of SRC as a public health issue. A recent analysis of ambulatory consultations between 2003 and 2013 documented a fourfold increase of reported diagnosis of concussion.2 Increased awareness and competencies within the healthcare professions are among the factors that might explain this increase in reporting. Yet, this epidemiological data most likely still represent an underestimation of the true incidence of concussion. Accordingly, further efforts are warranted to develop awareness around concussion for healthcare professionals, in sport and school environments, and in the general population.
One of the major initiatives to address the issue of SRC is the ongoing process of integrating knowledge into consensus statements and recommendations such as the Berlin Consensus on Concussion in Sport.3 The implementation of such recommendations has been beneficial at the highly resourced and structured elite levels of sport participation. However, their implementation across levels of participation, from grassroots to elite, with proper consideration for the broad spectrum of sport and leisure activities, environments (eg, school-based, non–school-based), available resources and access to qualified health professionals likely remains a challenge. This situation is illustrated by the fact that, following the Berlin consensus, the Canadian national strategy on concussion has focused most of its dissemination efforts at the elite level of national sport organisations.4 This allowed the development of sport-specific concussion management strategies that will hopefully facilitate further efforts at other levels of participation.
One remaining challenge is to achieve the implementation of best practices at every level of sport and leisure participation. This will likely rely on multifaceted strategies. Part of the solution might include legislation, such as the Lystedt laws adopted in the 51 US jurisdictions.5 Data on the implementation of these laws indicate that, although they contributed to increase awareness, measures of enforcement remain largely non-existent.1 6 In line with these findings, amendments have already been adopted in several States.7 In Ontario (Canada), a policy that outlined formal requirements about concussion in high schools was implemented in 2015.8 However, very limited information regarding the impact of this policy is available.9 More recently, the first legislative requirement related to SRC in Canada was adopted by the Ontario legislature.10
Challenge of global implementation
A key characteristic of additional implementation strategies should be to support a reflective process that prompts consideration of the challenges of knowledge transfer in a broad range of environments.11 Reflective learning, or the acquisition of knowledge with reflection about its context of application, is acknowledged as an important component of education on health-related issues.12 Relevant considerations for knowledge implementation related to SRC include the characteristics of the environment and sport, identification of barriers and tailoring of the implementation strategy.13
One key consideration was captured in a recent joint statement from the national family medicine and sport medicine organisations in Canada: “… key aspects of concussion prevention, detection and management occur prior to as well as after the initial medical intervention. … Physicians with expertise in sport and exercise medicine can contribute to these strategies by working in conjunction with families, schools, sports organisations, employers and governments to educate, support and empower the implementation of proper concussion prevention, detection and management protocols”.14 “Empower” is a key word in this statement because the availability of qualified healthcare professionals to directly manage the day-to-day aspects of concussion management in the sport environment, at home and through the return-to-school process will always be limited to all but the most resource-rich organisations. Outside of such environments, many aspects of concussion prevention, detection and management will depend on stakeholders in the sport, family and school environments. Multidisciplinary collaborations between healthcare professionals that are defined by the terms of medically supervised protocols have been proposed as a potential solution.15 For example, if timely access to medical experts on concussion is limited, a protocol could define how a school personnel, nurse or team therapist can play a role in the management of concussion, including timely referral for medical evaluations.
A potential strategy to improve access to concussion education for every possible stakeholder is the use of a massive open online course (MOOC). The term MOOC was used for the first time in 2008 and, over the last decade, with the availability of the World Wide Web, MOOCs have become a knowledge dissemination strategy used by many major universities.16 MOOCs are defined by several key characteristics: (1) a topic of broad public interest; (2) free and open registration; (3) online, self-paced delivery; (4) interactive educational support; (5) formal assessment.16 This strategy has been used in the field of sport and exercise medicine to promote the benefits of physically active lifestyles17 but, to our knowledge, had not been used to address the issue of concussion. By providing free access to education outside of formal educational programmes, MOOCs have the potential to increase the health literacy of the public and provide an innovative teaching model for learning focused on patient-centred and family-centred care.18 19
Accordingly, the assumption driving the development of a MOOC on concussion by Laval University (Quebec, Canada) was that it would successfully create a learning environment that brings together participants from every stakeholder group concerned with concussion regardless of the context of sport and leisure participation. The initial four iterations of the MOOC on concussion have been presented between April 2016 and October 2018. The current study had two main objectives: (1) to document the volume of successful participation and (2) estimate the impact of the course on protocol implementation. The analysis also provided insights regarding how to optimise the integration of an evaluation strategy in future adaptations of the MOOC.
Methodology
Structure of the MOOC
The MOOC was developed in French language on the ‘MonPortail’ learning and content management platform developed and owned by Laval University (Quebec, Canada).20Table 1 summarises the reference framework used to develop the educational structure of the MOOC. This framework is based on the recommendations of the Canadian Concussion Collaborative (CCC) regarding the objectives of a concussion management protocol (CMP).15 A scientific committee of eight persons representing relevant patient and public stakeholder groups reviewed and approved the objectives of the course (for details, see online supplementary additional material). Each of six modules includes two main sections: (1) the primary learning content and (2) an optional reflective process to support the adaptation of a CMP to each participant’s context.
Supplemental material
Supplemental material
Structure of the MOOC on concussion
This course was developed as part of the Laval University ongoing MOOC programme21 in areas of expertise, such as concussion, where a MOOC was felt to have potential for success. The course was offered four times between April 2016 and the fall of 2018, and relevant new developments (eg, Berlin consensus3 and Canadian guideline on concussion in sport4) were integrated prior to each edition. Returning participants can therefore use this course as a resource for updates on concussion.
The reflective process proposed for developing a CMP was adapted from the ‘Road map’ published by the CCC as a support for adapting and implementing such protocols.22 The participant’s role (eg, parent, coach, teacher, health professional), the specificities of the sport or activity, the characteristics of the environments (eg, school-based, non–school-based) and the available resources (eg, coach, healthcare provider) are examples of the factors that participants were invited to consider as part of the reflective process. Discussion forums were available in each module to allow participants to share experiences, comment on the contents of the different modules, and ask questions or request clarification. Forum participants were free to use their true identity or a pseudonym. The course instructor (PF) monitored the forum daily and provided feedback, additional information and answers as needed. The process for the evaluation of participants’ knowledge following each module and for the final evaluation was a set of 10 multiple choice questions randomly generated from a pool of 3 options for each question.
Patient and public involvement
Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research.
Participants and data source
The only requirement to participate was to provide a name and a valid email address. The terms of use of the MOOC included a specific mention that de-identified information would be collected and could be used to evaluate the utilisation of the site. The tracking functions of the ‘MonPortail’ platform provided de-identified information including number of registrations, number of accesses to each module and number of distinct participants accessing each module. Statistics on participation and success rates for each component of the evaluation were also available. A ‘Registrant’ was defined as a person who proceeded to register for the course regardless of subsequent participation. A ‘Participant’ was defined as a registrant that logged into the course at least once over the duration of the course. ‘Successful completion’ was defined as achieving an overall 60% success mark for the evaluations (success for each individual module was not required). Individuals who achieved a successful completion were eligible for a certificate of achievement. Optional survey questions were also used through the modules to document the collective profile of the participants (eg, sport, education, healthcare) and their intent to update or implement a concussion protocol. The forums also provided quantitative and qualitative data on the volume of interaction with and between participants. Finally, following completion of the course, participants were asked to complete an optional satisfaction survey. The Laval University ethical review committee considered that the data presented in the study meet the criteria for “research that relies exclusively on secondary use of anonymous information” and, therefore, did not require ethical review (Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans; Canada, 2018).
Results
The data regarding registration, participation and successful completion are presented in table 2. Among the 8368 registrants, 43%, 26%, 16% and 15% were participants of the first, second, third and fourth iterations, respectively. The proportion of registrants achieving ‘successful completion’ was consistent over the four iterations (range, 26%–32%), with 30% of all participants being eligible to obtain the attestation of success. Forty-three per cent of the “participants that accessed the course at least once” achieved successful completion.
Registration, participation and successful completion
An optional survey completed by 36.6% of the registrants was used to estimate the proportion of participant from different stakeholder groups (table 3). Over the four iterations of the course, “healthcare professional other than MD” consistently represented over half (54.6%) of the participants, whereas physicians represented only 4.2%. Coach and sport administrator was the next most prevalent group, with 10.1% of the participants. Education stakeholder, parents and persons affected by a concussion each represented from 3% to 7% of the participants. The answer “Other (not specified)” was selected by 14.8% of the respondents, but the functionalities of the course platform did not allow to collect free-text specifications with the answers.
Answers to optional survey question: “what category defines best your perspective or interest as a participant in this course?”
Another optional survey, completed by 1471 responders (17.6% of registrants or 24.8% of participants), was used to assess the intent of the participants to implement or update a CMP (table 4). The results indicate that 579 individuals (6.9% of registrants; 9.8% of participants; 39.4% of survey responders) across the four iterations of this course intended to update or implement a CMP in their sport or leisure environment.
Answers to an optional survey question about the participant’s intent to update or implement a concussion protocol
The volume of activity in the discussion forums is summarised in table 5 and demonstrates that about 15%–18% of participants used the forums. Key information from the optional satisfaction survey that followed each iteration is summarised in box 1.
Volume of discussion activity and access by participants in the forums*
Key information from satisfaction survey following the four iterations of the MOOC (number of respondents: 1023).
87% declared spending 1 to 3 hours per week (over 6 weeks).
Over 90% declared that the workload and duration of the course are adequate.
Global satisfaction for each module was rated as ‘highly appreciated’ or ‘appreciated’ by over 90% of participants.
A minority (23%) wished there were more opportunities for interaction between participants.
58% were satisfied or highly satisfied by the interaction with the educational support team (39% “did not know” or answered “does not apply”).
Over 90% felt the access to the course was easy, navigation was intuitive and answering the survey questions was easy.
96% of participants who used the forum answered that it was easy to use.
Discussion
Participation and completion
Registration rates for MOOCs are often impressive, but data related to participation and completion are more relevant from an educational or knowledge transfer perspective. Although criteria used to define eligibility for a ‘Statement of accomplishment’ vary across studies and are sometime not reported, the data allowing to establish the proportion of registrants that meet these criteria are generally reported and allow to compare participation between MOOCs.22 23 In this study, the criteria for successful completion was consistently met by about 30% of the 8368 persons who registered for the course and 43% of the participants who accessed the course at least once. By comparison, an analysis of nine MOOCs offered by the Johns Hopkins School of Public Health showed that registration ranged from 11 000 to 101 000 participants while the proportion that reached the criteria for a ‘Statement of accomplishment’ ranged from 4.2% to 14.9%.22 Based on a review of information available about registration in 91 MOOCs and completion for 42 MOOCs on a broad range of topics (not limited to health), Jordan24 found an average MOOC registration of around 43 000 students (range, 4500–226 000), with an average completion rate of 6.5% (range, 0.9%–36%). Recently, a completion rate of 38% for a MOOC on dementia was reported.18 We are not aware of published data from MOOCs on sport and exercise medicine topics. Nevertheless, in terms of successful completion rate, the French-language MOOC on concussion is in the upper range by comparison with other published MOOC experiences.
A gradual decrease in the total number registrants was observed over the first three iterations of the MOOC with a subsequent stabilisation at about 36% of first offering. This may indicate a gradual saturation of the reach for potential new participants. Among the considerations that might help define the optimal frequency of delivery of a MOOC on concussion is the 4-year cycle for the update of the international concussion in sport group recommendations.3
Over the four iterations, although only 15%–18% of registrants accessed forum activities, 23% of responders to the satisfaction survey reported that they wished there were more opportunities for interaction between participants. This suggests that additional strategies to stimulate interaction between participants could be considered in the future.
Factors contributing to completion
Beyond the public awareness about concussion and the associated motivation to gain knowledge on this topic, the success of the MOOC in terms of successful completion is likely explained by multiple factors that have been described in the growing literature on MOOCs. Karsenti25 has identified three strategies that likely contribute to completion: (1) mobile learning, (2) ‘gamification’ of learning and (3) adaptive learning. While the ‘gamification’ of learning was not exploited, mobile learning was present in the form of flexible formats (eg, video or podcast format), short learning units (±3–10 min), limited workload (1–3 hours per module), short overall duration (6 weeks) and discussion forums that allowed for interaction regarding a broad range of concussion-related issues. Together, combined with the flexibility allowed by the online nature of the course and current technological options to access the web, these mobile learning strategies likely contributed to the satisfaction and completion rates reported.
Adaptive learning is also a strategy found in the MOOC in the form of a basic learning profile, combined with an optional reflective process around the adaptation and implementation of a CMP in the specific setting of the participant. Considering that over 70% of the respondents (see table 3) reported being healthcare, school or sport stakeholders, this optional learning pathway might have been a factor that motivated sustained participation for many registrants. On the other hand, as illustrated by 61% of the survey respondents declaring that updating or implementing a protocol was not an objective (table 4), participants such as parents, victim of a concussion or medical doctors might have chosen not to participate if the entire focus of the course had been on protocol implementation.
Potential reach of an English MOOC on concussion
Of the 8368 persons who registered to the 4 iterations of the French-language MOOC, 70.0% were from Canada, 6.8% from Haiti, 6.1% from France, 11.2% from countries of the African continent and 5.8% from other countries. Data from the 2016 census estimated the Canadian population who declared ‘knowledge’ of English, French or both official languages as 23.8, 4.1 and 6.2 million, respectively.26 Assuming a proportional representation of potential participants, this indicates that an English adaptation of a MOOC on concussion has the potential to reach over three times more participants in Canada. The estimation of the potential reach of an English-language MOOC on concussion should also take into consideration the potential for registrations by international participants with a global English-speaking population that is more than five times larger than French-speaking.27 Based on the experience of the French-language MOOC on concussion, the University of Calgary and Laval University have developed an English adaptation of a MOOC in concussion with the assumption that it could reach three to four times more registrants.28
Limitations and potential for improvement
The integration of an evaluation strategy should be part of the development of MOOCs.19 29 Proposals for the design of MOOCs include multiple evaluation components: (1) pretesting as a comparative reference for subsequent evaluations throughout the course, (2) evaluation of both conceptual (measures of ‘what’) and procedural (measures of ‘how to’) learning, and (3) validated outcome measures to enable comparison across MOOCs.19 In the French-language MOOC, the evaluation of each individual module had a predominant focus on conceptual knowledge, while the final evaluation had a predominant focus on procedural knowledge based on clinical vignettes. However, the absence of comparative data from a pre-test evaluation did not allow us to assess knowledge increases.
Another limitation of the French-language MOOC experience is the limited capacity to directly evaluate its impact in terms of knowledge transfer and implementation. Beyond the fact that 579 participants intended to implement or update CMPs in their setting, and the finding that 2538 participants met the criteria for successful completion, no direct measure of knowledge improvement or subsequent implementation of CMPs was included in the evaluation strategy. To document such outcomes, a future MOOC would need to obtain consent from participants to take part in subsequent data collection processes.
Conclusion
In summary, the experience of four iterations of a French-language MOOC shows promising results supporting the use of MOOCs as an innovative educational strategy that may be part of the solution to the public health issue of concussion. The development of future MOOCs should include a comprehensive evaluation strategy to document the impact on knowledge and, if applicable, measures of knowledge implementation.
Acknowledgments
The authors would like to acknowledge the following persons for their contribution to specific elements of content and/or multimedia development for the MOOC in concussion: Jérôme Bourgoin, Stéphane Cayer, Isabelle Cossette, Julie Dionne, Isabelle Gagnon, Pierre Langevin, Natalie LeSage, Éric Martel, Bradford J. McFadyen, Caroline Poulin, Jean-François Proteau, Anne-Marie Provençal and Cyril Schneider.
Footnotes
Twitter @pfremo, @caemery@CarolynAEmery
Contributors PF was responsible for the development of the MOOC in concussion, data extraction and drafting the different iterations of the manuscript. All authors contributed to the analysis and interpretation of data, critical revision and final approval of the manuscript. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding This research was unfunded. However, Université Laval provided resources for multimedia development and the web platform used for the course.
Disclaimer Université Laval had no influence on the collection, analysis and interpretation of the data, or the writing of the report.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as online supplementary information. The de-identified data presented in this study were extracted from the Laval University educational web platform used for the MOOC in concussion. To protect the identity of individual participants, only the course instructor (PF) has access to this original data source.