Article Text
PDF
Abstract
Objective Identify all Sport Concussion Assessment Tool (SCAT2/3) studies, compare baseline and postconcussion results.
Design Systematic review (qualitative synthesis, weighted means).
Data sources 18 databases, 9 grey literature resources searched for SCAT2/3 data; 9150 articles identified, titles/abstracts assessed/data-entry independently by two reviewers.
Eligibility criteria for selecting studies Any studies reporting partial/complete SCAT2/3 data.
Results 21 studies with data (partial/complete data 16 SCAT2 (4087 athletes); 5 SCAT3 (891). Newcastle-Ottawa risk-of-bias scale: studies with maximum possible score of 4, 85% scored 3 or 4; studies with maximum possible score of 6, 75% scored 5 or 6. SCAT2 high schoolers: weighted mean score for symptoms 18.46 (22=no symptoms), Balance Error Scoring System (BESS) 26.14, Standardised Assessment of Concussion (SAC) 26.00 and SCAT2 total 88.63. Collegiate/adults weighted means: symptoms 20.09, BESS 25.54, SAC 27.51 and total SCAT2 91.20. Between-study and within-study variability similar to those of the high schoolers. Limited variability between genders. Only 2 studies report baseline and postconcussion scores and 9 partial scores, but data are too limited to provide weighted average scores.
Conclusions Group mean baseline SCAT scores for high school and collegiate athletes are similar, with minimal gender differences; baseline symptoms show more variability than other components. There are minimal data for elementary students and professionals, no data for adult non-collegiate athletes. Two studies provide preconcussion and postconcussion scores. No data on minimal significant clinical differences to guide players/coaches in withdrawing from a game in progress and deciding when recovery is complete and play can be resumed. The SCAT needs supplementing with clinical and neuropsychological return-to-play assessments.
- Concussion
- Measurement
- Evidence based review
- Sports
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
What is already known?
The Sport Concussion Assessment Tool (SCAT) is the most widely concussion assessment test, but before this review no summary data were available about SCAT component scores or between-sample and within-sample heterogeneity.
What are the new findings?
21 studies provided SCAT 2/3 assessments and included 4978 athletes (1067 females, 3831 males and 80 gender not stated).
Only one study reported SCAT data for a professional league (Finnish ice hockey players), there are minimal data on elementary students and no data on adult non-collegiate athletes.
Each player of a contact sport should complete the SCAT preseason to establish a baseline score.
The studies with data on preconcussion and postconcussion SCAT scores are too limited to provide weighted average scores. A study of rugby players during 62 games noted 52 concussions (44/1000 match hours) with 8 witnessed concussions, but 44 unwitnessed concussions identified postmatch by the King-Devick test.
Introduction
Effects of concussion
Concussion is typically conceptualised within the range of mild brain injury and may acutely affect cognitive functioning with amnesia, confusion, changes in attention, concentration, memory, information processing, learning and reaction time; cause headaches, dizziness, nausea, vomiting and affect vision, sleep and emotions.1–8
The American Medical Society for Sports Medicine9 estimated 3.8 million concussions occur in the USA annually, and 50% may be unreported. The rates of concussion per 1000 athlete exposures ranged from 0.17 to 0.23 in two reviews.9 ,10
Follow-up
The problems experienced by medical teams in follow-up are the often low percentages contactable several months after concussion,11–16 and thus we do not know the percentages of individuals who remain with consequences of concussion.
Measurement
In choosing a concussion measurement tool, it is important that it has excellent validity and reliability and measures concussion in a wide range of sports, age groups and both genders and be equally reliable at initial enquiry and follow-up.
It is important to assess all concussions promptly and prevent individuals returning to contact sports while concussed and especially important to assess subsequent concussions because a history of concussion is associated with a higher risk of subsequent concussions.9 ,17 Repeated concussions result in permanent structural changes/damage and lead to later brain disease.18 ,19
SCAT2/SCAT3
Several measures have been used to assess concussion: the Postconcussion Scale, the Sport Concussion Assessment Tool (SCAT), the Reporting Information Online, the Concussion Resolution Index and the Graded Symptom Checklist.20 The International Symposia on Concussion in Sport Zurich Consensus statement proposed the SCAT2 in 2009.21 The SCAT2 includes (1) a 22-item symptom checklist (score of 22=no symptoms), (2) a 2-item physical signs score (loss of consciousness, balance difficulties), a score of 2=no problems, (3) the Glasgow Coma Scale (no symptoms=score of 15), (4) the Maddocks Score (five questions about how oriented the player is to the current situation, and the score is not included in the overall SCAT2 score), (5) the Standardised Assessment of Concussion (SAC) (no symptoms=score of 30), (6) the modified Balance Error Scoring System (BESS) (score of 30=no errors), (7) a Coordination examination of five finger-nose movements (maximum score 1) and (8) a delayed recall of the component of the SAC which asks for repetition of six words (maximum score 5). The maximum SCAT2 score is 100. Each symptom can be rated 0–6 to produce a maximum symptom severity score of 132, which is not considered for the SCAT2 total score.
The 2012 Concussion in Sport Group (CISG) Consensus meeting concluded that there is good evidence to support the use of each of the components scored independently (symptoms, symptom severity, neurocognitive function and balance) but no evidence to support the use of a total score.22–28 The individual major components of the SCAT (Symptoms, Balance, SAC) have good validity and reliability. Owing to its wide relevance to many sports and ongoing development, the SCAT is likely to be the key concussion assessment tool and for these reasons is the focus of this review.
Purpose of this systematic review
Identify all Sport Concussion Assessment Tool (SCAT2/3) studies and compare baseline and postconcussion results.
Methods
Search strategy
The following databases were searched using predetermined search strategies discussed between the librarian, principal investigator and coinvestigators: MEDLINE, EMBASE, EBM Reviews (Cochrane Database of Systematic Reviews, ACP Journal Club, Database of Abstracts of Reviews of Effects, Cochrane Central Register of Controlled Trials, Cochrane Methodology Register, Health Technology Assessment, NHS Economic Evaluation Database), PubMed, PubMed Central, Web of Science, CINAHL, SPORTDiscus, Family & Society Studies Worldwide, Family Studies Abstracts and Health Source—Nursing/Academic Edition. In addition, nine grey literature resources were consulted, including Google, Google Scholar, OpenDOAR (http://www.opendoar.org), Health Sciences Online (HSO) (http://www.hso.info), ProQuest Dissertations & Theses, OAISter (http://oaister.worldcat.org), Sports Concussion Institute (http://concussiontreatment.com), Brain Trauma Foundation (https://www.braintrauma.org) and the Canadian Concussion Collaborative (http://casem-acmse.org/education/ccc). No limits on publication date were applied, and the search included studies in all languages and from all countries. All included studies were individually entered in the PubMed single citation matcher, and all relevant citation chains followed up to identify any relevant references.
Eligibility criteria
On the basis of the information in the abstract, studies had to provide partial or complete SCAT 2 or 3 data and 73 studies were read in full-text.
Data extraction and analysis
All study titles and abstracts were read independently and data extracted independently by two reviewers (RET, then either JA or RM) and included if the authors reported partial or complete SCAT2 or SCAT3 numerical outcomes.
Data analysis
For the high school and collegiate age groups and both genders weighted means (according to sample size) were computed for SCAT2 and its components. There were insufficient data for separate computations for elementary school children,29 professional athletes30 and SCAT 3 scores (table 3); and preconcussion and postconcussion scores were compared only for the two studies that provided them. Only one study included data for elementary students (age 9–10 years old)29 (table 2), one study provided data for players in a professional league30 (table 3) and the SCAT3 studies are too few to permit computation of weighted means; all these data are presented in the tables without further analysis.
Risk of bias assessment
To assess the overall risk of bias in the selected studies and permit stratification of studies by risk of bias, assessment was performed using the Newcastle-Ottawa Scale (NOS),31 developed to assess the quality of non-randomised studies in meta-analysis. Each study was assessed by two reviewers (RET then either JE or RM) with disagreements resolved by rounds of discussions. Each study was assessed for three components: the selection of the study groups; the comparability of the groups and the ascertainment of either the exposure or outcome of interest for case–control or cohort studies.
Results
Search
A total of 9150 articles were identified in all indicated databases and 77 from grey literature and a PubMed search in February 2016. After removal of duplicates, 1267 titles and abstracts which described assessments of concussion using the SCAT 2 and SCAT 3 were identified, 73 papers read in full-text and 21 studies included in this review (figure 1). Among the 21 included studies, 16 reported partial or complete SCAT2 data for 4087 athletes and 16 reported complete or partial SCAT 3 data for 891 athletes (tables 1 and 2, see online supplementary table S1).
SCAT2 scores at baseline
SCAT3 scores at baseline
PRISMA 2009 flow diagram. SCAT2 and SCAT3 scores at baseline and after mild brain injury/concussion: systematic review. SCAT, Sport Concussion Assessment Tool.
Risk of bias
No studies with interventions were identified and thus the columns for a hemi-cohort exposed to an intervention and the control hemi-cohort in the NOS31 are not filled (table 3). For studies with a maximum possible score of 4, 85% scored 3 or 4; and for those with a maximum possible score of 6, 75% scored 5 or 6.
Assessments of risk of bias in included studies, according to the NOS
Generalisability of the studies
The most representative study reported on 304 of all the 427 professional male ice hockey league players in Finland (the only study which reported professionals). Five studies focused on one sport: Echlin et al32 (67 junior hockey players), King et al33 (37 rugby union players), King et al34 (104 rugby union and rugby league players), King et al35 (50 rugby league) and Weber et al36 (32 wrestlers). Two studies combined reports on two sports: Jennings et al37 (71 footballers and baseball players) and Galetta et al38 (332 ice hockey and lacrosse players). Four studies reported individuals seen in concussion clinics.39–42 The rest reported participants in multiple sports with up to nine different sports.43
Variability within and between studies
Weighted mean SCAT 2 scores were similar for the high school and collegiate/university/adult non-university athletes (table 4).
Weighted mean SCAT2 scores by the age group
Symptom scores showed more variability between samples than the other SCAT2 components. For high school children, the lowest mean SCAT symptom scores (22=no symptoms) were 12.5 (Baker40) 16.4 (Mayfield)48 and 16.9 (Valovich McLeod)27 with the SDs ranging from 2 to 6.7. The lowest mean BESS score was 23.73 (Snyder and Bauer29 for those aged 9–10); SDs ranged from 2.0 to 3.58. The lowest mean SAC score was 24.2229 for those aged 9 and 10; SDs ranged from 1.9 to 3.01. The lowest mean SCAT2 total score was 84.35;29 SDs ranged from 3.7 to 7.55 (Snyder and Bauer29 for those aged 9 and 10), whereas in some studies (eg, Zimmer et al44) there was minimal variability. For collegiate/adult athletes between-study and within-study variability were similar to those of the high school children. For the two SCAT3 studies that provided complete data, Finnish ice hockey players30 had symptom scores of 20.5, BESS of 28.0 and SAC of 27.0 and for Benedict et al39 14, 24.1, and 27.
Variability in SAC scores
Three studies identified inaccuracy in repetition tasks that might account for variability in SAC scores. Among 13–15-year-old boys, 69% could repeat 4 digits correctly, 37% 5 digits, 12% 6 digits and 57% could say the months of the year backwards.45 Among 16–19-year-old males 79% could repeat 4 digits correctly, 37% 6 digits, 21% 6 digits and 70% could say the months of the year backwards.45 Among 12-year-old boys 85% repeated 4 digits correctly, 65% 5 digits, 31% 6 digits and 65% could say the months of the year backwards.43 Among 18-year-old men 100% repeated 4 digits correctly, 82% 5 digits, 72% 6 digits and 82% could say the months of the year backwards.43 Among college athletes (average age 20, range 17–32) SAC scores were higher than for high school students (eg, 51% for 6 digits and 92% months of the year).46 Female data were similar.46
Variability in scores before and after tasks without concussion
One study reported differences in symptoms after exercise (females symptom score 16.0 pre-exercise, females symptom score 11.5 after Leger 20 metre shuttle run test. Males symptoms score 18.7 pre-exercise, males symptom score 12.5 postexercise)47 while another study did not (total symptoms (21.56, 20.97), BESS (15.72, 18.81), SAC (27.72, 27.62) and SCAT2 total (93.06, 87.94)).36
Rates of detection of concussion
King et al33 found that during amateur rugby union matches five concussions were identified by a nurse at the pitch (average SCAT2 60.6, SD 4.2) but 17 were identified later by the King-Devick test with much higher SCAT2 total scores: (total SCAT2 84.2, SD 9.6). King et al34 during rugby union and league games noted eight witnessed concussions: total symptoms (8.6, SD 3.7), BESS (17.5, SD 3.4), SAC (23.0, SD 2.1) and 44 unwitnessed concussions with a difference only in those with lower BESS scores: total symptoms (8.6, SD 4.8), BESS (12.5, SD 6.2) and SAC 23.5, SD 3.1).
Studies of concussed players
supplementary table S1). Mayfield et al48 reported data from the day of concussion to the 10th day after concussion for 119 concussed athletes and found most improvement in symptom scores: baseline total symptoms were 16.54 (day of impact 10.9; day 3 14. 8; day 10 19.9); BESS baseline 27.0 (day of impact 24.5; day 3 26.0; day 10 27.4); SAC baseline 26.6 (day of impact 25.2; day 3 26.3; day 10 26.6) and total SCAT2 baseline 88.00 (day of impact 77.76; day 3 84.74; day 10 91.68). Putukian et al49 for 32 concussed players found their baseline Total Symptom Score was 19.44 and postconcussion 13.00, BESS 26.13 and 24.54, SAC 27.63 and 27.13 and SCAT2 total (excluding the Glasgow Coma Scale and physical signs) 74.16 and 65.28. The weighted averages for these two studies combined show an increase in postconcussion of 8 symptoms, a decrease of 2.3 on the BESS and an increase of 1 on the SAC. Nine studies report partial data for concussed players (see online supplementary table S1), and the incomplete preconcussion and postconcussion data are insufficient to provide reliable weighted average SCAT scores.
Gender differences
There were sufficient gender data to compute weighted mean SCAT2 scores only for high school students and these showed minimal differences between genders. For females weighted mean symptom scores were 18.44 (males 18.86), BESS scores 26.75 (males 25.89), SAC scores 26.76 (males 26.03), and total SCAT 2 scores 90.10 (males 88.79) (table 5).
SCAT2 scores: gender differences
Discussion
Amount of published SCAT2/3 data
This systematic review includes SCAT data from 4978 athletes. Most studies (n=16) provided partial or complete SCAT2 data, and five provided partial or complete SCAT3 data. There are limited data for elementary students, data only for one professional league and none for adult non-collegiate athletes. Generalisability was highest for a study of Finnish professional male ice hockey which assessed most players in the league30 and lowest for studies of individual schools or groups including many sports. SCAT component average scores were homogeneous between samples, but there was considerable heterogeneity within samples. Available studies suggest no strong differences between genders.
Limited data postconcussion
There are limited data on concussion. Only two studies (n=151) provided preconcussion and postconcussion data. A study of 104 rugby players during 62 games noted 52 concussions (44/1000 match hours) with 8 witnessed concussions, but 44 unwitnessed concussions which were identified postmatch by the King-Devick test.34
Preseason baseline SCAT scores
Group averages are not perfect scores on all SCAT components preseason because some athletes had pre-existing symptoms (eg, headache) or because some athletes had problems with test items (eg, repeating serial numbers backwards). Thus, it is important to test players at least preseason and also during the season to detect unwitnessed concussions.
Changes in SCAT scores during practice and play
There are only two studies which report how SCAT component scores change without a concussion when players exercise or become exhausted during play, and more studies are needed during game intermissions to assess the effects of play and exhaustion so that when a concussion occurs the total change after a concussion can be partitioned into the effects of play and concussion. General postevent testing (eg, King-Devick test) is important in identifying concussions which are missed or unwitnessed during gameplay.
There are insufficient data on the longitudinal normative and abnormal range of SCAT performance in athletes, and this limits its current use as a science-based diagnostic and prognostic tool to guide players and coaches when to stop players continuing during matches and when to allow them to resume their sport.
Strengths and limitations of this review
We searched 18 databases and 9 grey literature resources without limitation of language or date. We could not combine data from elementary, high school and collegiate players because of the differences in physical size, the multiple sports played and individual sports results not provided and likely differences in competitiveness and force of contacts. Instead, we report for each age group means weighted by the study sample sizes. The effects of heterogeneity within and between samples were thus not explored statistically. Owing to the limited amount of data for many sports and age groups, the absence of data for adult non-collegiate players and data available only for one professional league, we were not able to draw conclusions for these groups.
Conclusions
SCAT2/3 data are available for 4978 athletes, but there are minimal data for elementary school students, professional athletes and concussed athletes. There is no uniformity across sports in preseason SCAT testing to obtain baseline scores, numbers of observers to increase the likelihood of detecting concussions during play or using King-Devick tests to detect unwitnessed concussions. There is an urgent need for systematic adoption of continuous follow-up assessments including complete SCAT and King-Devick testing in all age ranges of amateur and professional athletes participating in sports with risk of concussion and follow them if concussed until fully recovered. Minimum clinically important differences in SCAT scores37 have not been defined and would need to be based on careful assessments by several assessors with minimal intraobserver and interobserver variability and detailed follow-up of athletes to determine full recovery and appropriate return to play. Until these problems have been solved, the SCAT should be used cautiously for clinical and return-to-play decision processes and be complemented with an extensive medical examination and quantitative neuropsychological testing providing objective performance decrement data.
Acknowledgments
The authors thank Dr Benedict and Carson for additional data.
References
Footnotes
Contributors RET conceived the systematic review, assessed titles and abstracts from the search, entered data into the review, computed data and wrote all drafts of the article. MMV conducted the searches. Titles/abstracts were assessed and data independently entered into the text and tables, and the text was edited and amplified by JA, RM and MMV. All authors agree with the final version of the article.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.