Article Text
Abstract
Objectives Effectively supporting the mental health of elite athletes and coaches requires validated tools that assess not only individual-level factors but organisational-level influences. The aim of this study was to develop a bespoke scale assessing perceived psychological safety within high-performance environments.
Methods 337 elite athletes (M=24.12 years) and 238 elite-level coaches and high-performance support staff (HPSS; M=41.9 years) identified via the Australian Institute of Sport provided data across a range of mental health and well-being domains. Exploratory factor analysis (EFA; n=169 athletes) with parallel analysis identified the Sport Psychological Safety Inventory (SPSI) factor structure. Confirmatory factor analysis (CFA) validated the identified structure in separate validation subsamples of athletes (n=168) and coaches/HPSS (n=238).
Results EFA identified the 11-item, 3-factor SPSI. Factors assessed domains of the Mentally Healthy Environment, Mental Health Literacy and Low Self-Stigma. All scale items loaded strongly on their specific domain. CFA model fit indices validated scale structure for athletes and coaches/HPSS. Internal consistency and convergent and divergent validity were evident. Logistic regression indicated that incrementally higher Mentally Healthy Environment scores reduced the likelihood of athletes scoring in the ‘moderate’ range of general and athlete-specific distress, with a stronger endorsement of the Low Self-Stigma subscale reducing the likelihood of being identified for athlete-specific distress.
Conclusion Psychometric properties of the SPSI support scale utility among athletes and coaches/HPSS in elite sports settings, though further psychometric efforts are needed. This brief measure may support benchmarking efforts across elite sporting contexts to improve mental health culture and broader well-being among athletes and coaches/HPSS.
- athlete
- mental
- psychology
- psychiatry
- young
Data availability statement
No data are available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Psychological safety is a widely applied and valued concept in organisational literature given its association with high-performing teams.
The concept of psychological safety has not been thoroughly applied to elite sporting environments.
WHAT THIS STUDY ADDS
The Sport Psychological Safety Inventory (SPSI) was developed and psychometrically validated as the first sports-specific measure of psychological safety for elite athletes, coaches and high-performance support staff.
Higher scores on the SPSI were associated with a lower likelihood of being in the moderate range for psychological (or athlete-specific) distress.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE AND/OR POLICY
The SPSI may assist sporting contexts in their efforts to benchmark and improve cultures of psychological safety.
Introduction
The prevalence of mental ill health in elite athletes appears comparable to the general population.1–3 However, rates can vary due to contextual factors,4 including measurement of symptoms during the performance cycle, career stage, gender and type of sport, including team versus an individual sport. Mental ill health is defined as threshold and subsyndromal mental disorders.5 In elite sports, risk factors for mental ill health include injury, performance anxiety, suboptimal performance, lack of non-athletic identity, selection pressure, perfectionism, financial insecurity and retirement.6 7 These risk factors are not unique to athletes. Coaches and other high-performance staff working in the daily training environment (eg, sports medicine practitioners, athletic trainers/physiotherapists, nutritionists) are also exposed to many of the stressors that operate within results-oriented cultures and environments.8 9
Available evidence suggests that athletes have more negative attitudes towards help-seeking than non-athletes.10 11 Commonly reported barriers to help-seeking include, mental health stigma, misunderstandings of mental toughness, and low mental health literacy.12 13 Mental toughness refers to high self-belief, self-reliance and an ability to pursue goal-directed behaviour despite adversity.14 The emphasis on mental toughness in elite sports can lead to unintentional consequences, such as a reluctance to disclose mental health symptoms or psychosocial difficulties (for fear of appearing weak) or to seek help and support.15 Cultures that prize mental toughness and ‘win at all costs’ attitudes may also reduce perceived psychological safety, whereby open disclosure of vulnerabilities—including mental health symptoms—is implicitly or explicitly discouraged or prohibited. Athletes also avoid disclosures of mental health vulnerabilities due to fear of career-related repercussions, including selection loss, competition opportunities or contract renewal.10 Each of these barriers may indicate low perceived psychological safety in elite sports settings. Each is underpinned by a reluctance to demonstrate perceived weakness (ie, taking an interpersonal risk) to avoid negative consequences.
Psychological safety refers to a sense of confidence in taking interpersonal risks or making mistakes without fear of negative consequences.16 Since its original application in organisational settings, research on psychological safety has been applied to various contexts, including healthcare, education, manufacturing and technology.17 Psychological safety is identified as a key determinant of high-performing teams requiring communication, trust and decision making under pressure.18 It is positively influenced by organisational policies and procedures,19 including inclusive and transformational leadership styles.20 Psychological safety and its association with mental health and well-being is a growing area of interest in elite sport.21 Vella et al suggest that psychological safety in sport is conceptualised as a group-level construct but perceived (and usually reported) at the individual or subjective level.22
The International Olympic Committee’s Mental Health in Elite Athletes Toolkit refers to psychological safety in sports settings as environments where athletes are comfortable being themselves, can take necessary interpersonal risks, have the knowledge and understanding of mental health symptoms and disorders, and feel supported and comfortable in seeking help if needed.23 In this sense, psychological safety may act as an environmental protective factor for mental ill health and facilitate early recognition of, and response to, symptoms of mental ill health. Mental health literacy is an essential component of managing and maintaining one’s mental health.24 In addition to increasing knowledge about common symptoms of mental ill health and available supports, mental health literacy programmes have been shown to impact factors that contribute to creating and maintaining psychologically safe climates, including reducing stigma, normalising mental ill health and increasing confidence and intentions to help others.25
The limited evidence base on psychological safety in elite sports suggests a positive relationship between psychological safety and team performance,26 fostering team resilience,27 and acting as a buffer against athlete burnout.21A recent expert Delphi study identified psychological safety as an important characteristic of resilient elite sports organisations.28 Given potential benefits in promoting help-seeking, well-being, resilience and team performance, psychological safety may be an invaluable construct to include in future programme evaluation activities in sports settings.29
Most studies investigating psychological safety in organisational settings apply Edmondson’s seven-item Team Psychological Safety Scale.16 This measure provides a global assessment of the social climate of a team/organisation but does not make specific reference to psychological safety in the context of mental health. Further, the Team Psychological Safety scale reported low reliability (α<0.70) and mediocre model fit in an elite sports setting,21 supporting the need for a bespoke elite sports measure. In their study, Fransen et al report on a modified six-item version of Team Psychological Safety Scale (α=0.70) reporting weak-moderate positive associations between identity leadership and team identification variables, and a weak negative association with athlete burnout. Given the questionable statistical validity of the Team Psychological Safety Scale in elite sport,21 we aimed to: (1) develop a bespoke scale tailored to measuring psychological safety in elite sporting contexts as it relates to mental health and (2) provide initial psychometric properties for the tool.
Method
Design
A cross-sectional online survey design was implemented. Participants provided data on a range of standardised mental health screening tools, in addition to items comprising a bespoke scale measuring psychological safety in elite sports.
Participants
Eligible participants were National Sporting Organisation (NSO) contracted elite athletes aged ≥16 years and coaches/high-performance support staff (HPSS, including sports medicine staff and other allied health professionals) identified on the Australian Institute of Sport (AIS) contact list. Other than age and ability to complete the survey in English, no exclusion criteria were applied.
Measures
Athlete-specific distress
The Athlete Psychological Strain Questionnaire (APSQ)30 31 is a 10-item measure of psychological distress specific to athletes (eg, ‘I could not stop worrying about injury or performance’). Items are scored using a five-point scale ranging from (1) ‘None of the time’ (2) ‘A little of the time’, (3) ‘Some of the time’, (4) ‘Most of the time’ to (5) ‘All of the time’, with total scores ranging from 10 to 50. The APSQ was administered to athletes only (but not coaches/HPSS). Cronbach alpha in the present sample was 0.87.
General psychological distress
The Kessler Distress Scale (K10)32 is a 10-item measure of general psychological distress, which assesses the frequency of reported common symptoms of anxiety and depression over the previous 4 weeks (eg, ‘During the last 30 days, about how often did you feel that everything was an effort?’). As per the APSQ, items are scored on a five-point scale ranging from (1) ‘None of the time’ to (5) ‘All of the time’, with total scores ranging from 10 to 50. Cronbach’s alpha in the present sample was 0.91.
Sport Psychological Safety Inventory
An initial item pool of 25 items (see online supplemental table 1) was developed in consultation with experienced mental health, sports psychology and sports medicine practitioners working across a range of elite and professional sporting codes and settings. The wording of Sport Psychological Safety Inventory (SPSI) items was refined for the AIS cohort following a prior iteration trialled in a professional sports setting. SPSI items were developed to assess a broad range of factors related to psychological safety in the elite sporting environment, with a particular focus on the IOC definition of psychological safety.23 These factors included disclosure safety, interpersonal risks, mental health literacy, logistics of help-seeking, stigma and perceived impact on career. A five-point response scale was used with items ranging from ‘strongly disagree’ (0), ‘disagree’ (1), ‘neutral/unsure’ (2), ‘agree’ (3) ‘strongly agree’ (4). Several SPSI items are reverse scored before analysis. Higher total scores indicate greater perceived psychological safety.
Supplemental material
Well-being
The Warwick-Edinburgh Mental Well-being Scale (WEMWBS)33 is a 14-item measure that assesses mental well-being over the previous 2 weeks. The WEMWBS includes items such as ‘I’ve been feeling good about myself’, where items are scored from ‘none of the time’ (1) to ‘all of the time’ (5). Total scores range from 14 to 70, where higher scores indicate higher mental well-being. There is no established cut-off score for the WEMWBS. Cronbach’s alpha in the present sample was .94.
Procedure
Data was collected between 16 March 2020 and 31 May 2020. The broader survey was designed in consultation with AIS Mental Health and Well-being staff. Recruitment procedures mirrored those previously reported for the 2018 AIS Mental Health Audit.3 Potential participants were contacted via SMS or email. The online survey was anonymous and was completed by participants at the place and time of their choosing.
Data analysis
Statistical procedures are outlined in full in online supplemental information and summarised below. The athlete sample was randomly partitioned into an athlete calibration sub-sample (n=169) and an athlete validation subsample (n=168). Exploratory factor analysis (EFA; principal axis factoring) with direct quartermin (delta=0) rotation was undertaken with data from the athlete calibration sample to identify the factor structure of the SPSI with data provided by the athletes. The item pool (see online supplemental table 1) for the SPSI includes several negatively valenced items, which were reverse scored. Parallel analysis was conducted to identify the number of underlying factors. Confirmatory factor analysis (CFA) was undertaken on data from the athlete validation sample and the coaches/HPSS according to the factor model established using the calibration sample. Reported indices were RMSEA <0.05 (indicating close fit via the p-close value), TLI and CFI ≥0.90 for acceptable fit and ≥0.95 for excellent fit, and SRMR ≤0.08. Bifactor analysis was subsequently undertaken to determine if the SPSI total score should be considered. Internal consistency for subscales of the SPSI was evaluated using Cronbach alpha (α) and McDonald omega (Ω) maximum likelihood coefficients, with coefficients ≥0.70 indicating acceptable reliability. SPSI percentile distributions were examined separately for athletes and coaches/HPSS, with those scoring ≤25% quartile (eg, low perceived psychological safety) examined on distress and well-being outcomes. Multivariate analysis of variance with follow-up univariate tests examined mean differences between athletes and coaches on SPSI domains. Spearman correlations were examined between outcome variables. Generalised linear regression examining binary logistic models (α=0.01, 99% ORs) explored the three SPSI domains as predictors of K10 (≥16) and APSQ (≥15) moderate distress cut-off thresholds. SPSS V.26.0 and Mplus V.8.2 were used for analyses. Reporting is consistent with the STROBE Statement for observational (cross-sectional) studies.
Patient and public involvement
A steering committee (including athlete representatives) was established. This group provided feedback on the structure and overall methodology of the larger survey.
Results
Survey responses were received from 327 athletes, representing a response rate of 16.5% of all NSO contracted athletes in Australia. In addition, responses were received from 238 coaches/HPSS, representing a response rate of 29.6% of all NSO contracted coaching and performance staff in the Australian NSO system. Demographics, help-seeking and percentage of participants meeting thresholds for moderate distress are presented in table 1. The two athlete subsamples were statistically equivalent on all demographic variables. On average, athlete participants were aged in their mid-20s, were predominantly heterosexual, single or partnered, and spent approximately 2.5 months travelling for their sport in the last year, with most (67%) identifying as female. A total of 15% of athlete participants reported currently accessing formal mental health support, with 61% indicating they had never accessed mental health support. Just over half of athlete participants scored in the moderate range for athlete-specific distress (APSQ ≥15; 55.8%) and general psychological distress (K10 ≥16; 51.3%). Coaches/HPSS were on average aged in their early 40s, were predominantly heterosexual, were likely to be married, with 40% scoring in the moderate range for general distress.
Exploratory factor analysis
The KMO value (0.871) and Bartlett’s test of sphericity (p<0.001) indicated excellent factorability of the data. Candidate items exhibited mild negative skew (see online supplemental table 2) but did not exceed criteria requiring transformation (eg, ≥2.0).34 35 Parallel analysis was undertaken with the full pool of 25 items, indicating a six-factor solution, which was subsequently imposed on the data. This initial analysis yielded factors with eigenvalues ranging from 7.86 to 1.05, accounting for a cumulative total of 56.65% of scale variance, converging in 17 iterations. Fourteen items (items: 15, 16, 14, 13, 4, 19, 20, 21, 22, 24, 23, 25, 18, 17) failed to either load at the threshold of ≥0.56 or loaded as a single or dual item factor. These items were subsequently omitted. Parallel analysis was reiterated on the remaining items, indicating a three-factor solution (KMO=0.842, Bartlett’s p<0.001). This 11-item, 3-factor solution was parsimonious, accounting for a cumulative total of 64.36% of scale variance, converging in six iterations. There were no cross-loading or underloading items or any two-item factors. Factor 1 (eigenvalue=4.24) was named Mentally Healthy Environment and accounted for 40.22% of variance after rotation, Factor 2 (eigenvalue=1.81) was named Mental Health Literacy and accounted for 16.42% of variance after rotation and factor 3 (eigenvalue=0.85) was named Low Self-Stigma and accounted for 7.72% of variance after rotation. As shown in table 2, loadings for each factor were strong, with clear measurement differentiation between the factors (ie, very low loadings on corresponding non-identified factors).
Confirmatory factor analysis
Using data from the calibration sample of athletes, the EFA three-factor solution was subject to CFA. For athletes, the unidimensional model (all items loading on a single factor) was a poor fit for the data, with all indices failing to meet recommended cut-offs (see table 3). In contrast, the second-order hypothesised model indicated a good fit (see table 3), although the SRMR (0.081) was marginally above the recommended cut-off of 0.80. Finally, the bifactor model indicated an excellent model fit on all indices. However, while the bifactor auxiliary indices indicated that the percentage of uncontaminated variance (PUC) index was relatively high (PUC=0.73), the estimated common variance (ECV) index for the total score fell below the recommended cut-off of 0.70 (ECV=0.33), as did the average relative parameter bias (recommend <0.15; ARPB=0.41). Standardised loadings are reported in figure 1.
For coaches/HPSS, the unidimensional and second-order models were poor-mediocre fits to the data, whereas the bifactor model indicated excellent model fit. As per the athlete analysis, the PUC index was relatively high (PUC=0.73), and ARPB was within the accepted range (.12). However, the ECV index for the total score was relatively low (ECV=0.50). The bifactor model reported an excellent fit for the combined sample, with ARPB within the accepted range (0.070) and the ECV=0.46.
SPSI properties
Online supplemental information and online supplemental table 2–5 provide evidence of SPSI internal consistency, percentile distributions, between-group differences (eg, higher scores observed for coaches/HPSS) and SPSI convergent and divergent validity.
SPSI domains and predicting general and athlete-specific distress
Generalised linear modelling (binary logistic) was used to predict categorical outcomes of moderate distress on the K10 (for both athletes and coaches/HPSS) and APSQ (for athletes only) using the three SPSI subscales in addition to past 12 month mental health treatment (yes/no). For the K10, the overall model was significant χ2(4)=54.38, p<0.001, with the Mentally Healthy Environment subscale (OR 0.829, 99% CI .744 to 0.914, p<0.001) significantly predicting general psychological distress caseness. A one-unit increase in Mentally Healthy Environment scores was protective against (eg, decreased the likelihood of) being in the K10 Moderate range by 17.1%. For the APSQ, the overall model was significant χ2(4)=49.50, p<0.001, with the Mentally Healthy Environment subscale (OR 0.872, 99% CI 0.812 to 0.944, p<0.001) and the Low Self-Stigma subscale (OR 0.836, 99% CI 0.756 to 0.925, p<0.001) significantly predicting athlete-specific psychological distress caseness. A one-unit increase in the Mentally Healthy Environment subscale was protective against (eg, decreased the likelihood of) being in the APSQ Moderate range by 12.5%, and a one-unit increase in the Low Self-Stigma subscale was protective against (eg, decreased the likelihood) of being in the APSQ Moderate range by 16.4%. For coaches/HPSS, the overall model for psychological distress (K10) was significant χ2(4)=36.06, p<0.001, with Low Self-Stigma, significantly predicting psychological distress caseness (OR 0.841, 99% CI 0.722 to 0.978, p<0.001). A one-unit increase in Low Self-Stigma was protective against (eg, decreased the likelihood of) being in the K10 Moderate range by 15.9%.
Discussion
The current findings support the initial psychometric properties of the SPSI in a large sample of elite athletes and coaches/HPSS working in Australia’s elite sporting environments. Results suggest the potential utility of the SPSI in identifying environments where psychological safety relating to mental health is comparatively low, which may be associated with worse indices of overall mental health and well-being.
SPSI factor structure
Factor analyses supported the three-factor structure of the SPSI, assessing domains of Mental Healthy Environment, Mental Health Literacy and Low Self-Stigma, each of which was positively inter-correlated for both athletes and coaches/HPSS. The Mentally Healthy Environment domain focuses on perceived organisational support for mental health problems among those in the sporting system. The Mental Health Literacy domain can be considered an integral component of mental health-related psychological safety,24 as this knowledge base is fundamental to recognise symptoms of mental ill health and the need for support. The Low Self-Stigma domain assesses an individual’s perspective related to mental health self-stigma within the sporting context. For athletes, each of the SPSI domains was negatively associated with general and athlete-specific distress (eg, lower psychological safety was associated with greater distress) and positively associated with well-being. For coaches/HPSS, significant negative associations were observed between each SPSI subdomain and general psychological distress (K10).
Utility of the SPSI and implications
Logistic regression analyses predicting moderate distress caseness indicated that high scores on the Mentally Healthy Environment and Low Self-Stigma subscales were associated with lower likelihood of experiencing psychological distress (or athlete-specific distress) of at least moderate severity. It was particularly noteworthy that scores on the Mental Health Literacy subscale were not predictive of distress, suggesting that awareness of mental health problems and symptoms is insufficient in protecting stakeholders in the elite sports environment from distress. Instead, more important in reducing the likelihood of psychological distress is the broader culture related to perceived organisational support for disclosure of mental health problems and proactive support for their management. Consistent with Vella et al’s concept analysis of psychological safety in sport being a group-level construct,22 this ‘culture’ could be perceived by the individual as reflecting their teammates, their team and/or their sport as a whole. This finding serves as a ‘call to arms’ for sporting organisations to actively engage in the process of creating and implementing the systems and environments that support safe disclosures and management of mental health symptoms. Equally, it also arguably requires sporting organisations to demonstrate a genuine commitment to the mental health and well-being of key stakeholders in the sporting system.6 For example, the AIS has implemented Well-being Health Checks. A key aspect of these checks is that they assess how well athletes, coaches and staff are supported by their National Sports Organisation when it comes to a culture of feeling safe to disclose mental health difficulties. The organisation proactively supports them to manage these difficulties.
Evidence from organisational literature supports a range of positive outcomes associated with psychologically safe teams, including facilitated learning task and team performance, engagement and creativity.20 These benefits are thought to occur because psychological safety allows teams to provide open feedback, discuss errors, collaborate and experiment with new ideas.17 Psychological safety may also act as an environmental protective factor against mental ill health,36 37 particularly due to promoting openness and vulnerability between team members. Appropriate and supportive responses to mental ill health disclosures also contribute to psychologically safe climates, as these responses demonstrate that it is safe to express vulnerabilities. Exposure (through observation or direct experience) to adverse reactions following disclosure is informative about cultural norms or boundaries of acceptable behaviour, preventing subsequent help-seeking.38 39 Indeed, athletes report that supportive responses from sport-related networks, particularly coaches, facilitate help-seeking.10
Limitations and future directions
While this study offers a new measurement tool developed with robust methods, several important limitations and future directions should be considered. The response rate from the participating NSO’s suggests that the present sample may not be representative. The period of data collection co-occurred with the early stages of the COVID-19 pandemic and associated uncertainty regarding the occurrence of the Tokyo 2020 Summer Olympics and Paralympics, which were subsequently rescheduled to 2021. This may have increased psychological stressors experienced by participants. Future studies should extend the analysis of convergent and divergent (and predictive) validity, including existing measures of psychological safety (eg, the Team Psychological Safety Scale)16 and other outcomes that would be expected to be associated with the SPSI (eg, established measures of mental health literacy, self-stigma, group level and organisational level drivers). Future work should look to validate cut-off scores for the SPSI. Further, results of the bifactor analysis suggest some caution may be warranted in using the SPSI total score (as it accounted for between 33% and 46% of common variance), below the conservative criteria of 70%.40 Accordingly, we call for replication studies of the psychometric properties of the SPSI in larger and more diverse samples (eg, cross-cultural settings).
Conclusion
The SPSI offers a brief tool for sporting organisations to assess perceptions of psychological safety among their workforce. Use of the SPSI could enable benchmarking and identification of change over time.
Data availability statement
No data are available.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by University of Melbourne Human Research Ethics Committee (ID: 1442705.12). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We thank all study participants for their time and willingness to be involved in the study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @CC_Walton
Contributors SR, RP and LO conceived the idea for the study. RP, SR and VP were responsible for the survey design and survey implementation. MB and MC contributed to survey design. AK, ML, RP, LO and SR developed the initial item pool for the SPSI based on an earlier iteration, and this was refined with the assistance of MB and MC. SR conducted the analyses as raw data was not shared with any authors beyond Orygen. SR drafted the initial manuscript. SR accepts full responsibility for the work and conduct of the study, had access to the data, and controlled the decision to publish with consent of all co-authors. All authors contributed to the data interpretation and revision of the final manuscript. The authors read and approved the final manuscript.
Funding Australian Sports Commission, Australian Institute of Sport. Grant number not applicable.
Competing interests Authors MB and MC are employed by the Australian Institute of Sport, which is funded by Sport Australia.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.