Discussion
This study aimed to assess Open Science practices within physical activity behaviour change intervention reports. It was found that Open Science practices varied among the assessed 100 physical activity behaviour change intervention reports. Most reports were open access and pre-registered, with reported funding sources and conflicts of interest. However, open materials, data and analysis scripts were not frequently provided and no replication studies were identified.
Pre-registration of studies was found to be slightly more common for physical activity intervention RCTs (78%), than found in smoking cessation intervention RCTs (73%)22 and much more common than in wider psychological research of varying study designs (3%).21 In our study, similar amounts of studies were pre-registered prospectively (55.7%: prior to data collection commencing) or retrospectively (44.2%: after data collection had commenced),30 although this distinction between pre-registrations has not been assessed in comparable research. The common prevalence of retrospective pre-registration via clinical trials is arguably not true pre-registration, nor transparent from the study’s outset.12 13 One included study was noted as a Registered Report,39 where in-principle acceptance to journals is given based on study proposals at conception stage, rather than based on completed studies and their reported findings.44 45 No Registered Reports were identified in smoking cessation22 and psychology,21 perhaps reflecting a slow increase in Registered Report numbers over time.15 Protocols were available as separate papers or linked publications in 41% of included physical activity studies, which is higher than in smoking cessation studies (29%)22; and wider psychology research (0%).21 The increased prevalence of protocols within physical activity and smoking cessation likely reflects greater availability of health-related protocol publications,46 via specific journals such as JMIR Research Protocols and via protocols as specific types of publications within wider journals such as BMC Public Health and Trials. High prevalence of protocols in this study is also indicative of RCTs being both a common study design in health and intervention research47 and a study design typically accompanied by research protocols.48
Open access reports were at similarly moderate levels in physical activity (73%) than in smoking cessation (71%)22; and psychology (65%),21 but greater than the 45% observed in the social sciences,20 the 45% across scientific literature published in 201516 and the 25% in biomedicine.23 This high rate of open access publishing in physical activity interventions may reflect increasing requirements by health funding bodies for open access publications,49 as well as increasing usage of preprint servers such as medRxiv for medical sciences and PsyArXiv for the psychological sciences.50
Open materials were less commonly available in physical activity reports (8%) than in smoking cessation reports (13%),22 psychology (14%)21; and biomedicine (33%).23 Open data were also less common across physical activity reports (4%) than in smoking cessation reports (7%),22 but greater than the 2% of wider psychological research.21 Provision of raw data as supplementary files to published intervention reports or via trusted third-party repositories such as the Open Science Framework is important to facilitate evidence synthesis. Open analysis scripts were found to be as infrequently provided in physical activity studies than in smoking interventions and wider psychological research (all 1%).21 22 No replication attempts were identified in this sample of physical activity intervention reports, same as within smoking cessation reports22 but less than in the social sciences (1%)20; and in wider psychology studies (5%).21
Declaration of funding sources were declared in physical activity reports (93%) similarly to smoking cessation reports (95%)22; more so than wider psychology (62%),21 social sciences (31%)20 and biomedical science reports (69%).23 Similarly, a conflict of interest statement was provided as commonly in physical activity reports than in smoking cessation reports (88% in both)22 and higher than in wider psychology (39%),21 social sciences (39%)20 and biomedical sciences reports (65%).23 Eight per cent of studies reported conflicts from private companies including activity, pharmaceutical and other companies, less than the 20% of studies reporting company funding in smoking cessation interventions.22
Future steps to increase Open Science in physical activity interventions
This research has demonstrated a need to address the low levels of Open Science engagement in physical activity research, particularly in the areas of open materials, data, analysis scripts and replication attempts. As with any complex behaviour change, this transformation requires systems change across bodies involved in the development, running and publication of physical activity research: researchers, research institutions, funding organisations, journals and beyond.1 9 In order to develop effective behaviour change interventions, it is important to use a systematic and comprehensive approach to intervention development, underpinned by a model of behaviour and theoretically predicted mechanisms of action.51–53 The Capability, Opportunity, Motivation, Behaviour model54 posits that changing behaviour involves changing one or more of the following: capability (psychological and physical capacity to engage in the behaviour), opportunity (external factors that make the execution of a particular behaviour possible or prompt it) and motivation (internal processes that energise and direct behaviour). We argue that understanding the capability, opportunity and motivation associated with Open Science practices9 and developing interventions to address these determinants of behaviour change,55 is key to increase engagement with Open Science.
For example, low perceived capability towards Open Science practices in physical activity researchers can be addressed by providing researchers with training tailored to the context of activity intervention research (eg, online training on how to make anonymised activity monitor data openly available, how to use preprint servers most relevant to activity research or how to make their activity analysis reproducible). Opportunity to engage in Open Science practices can be facilitated within institutions, encouraging discussions around Open Science in the context of physical activity research19 and in science more broadly,21 23 56 as well as developing a research culture valuing and promoting the benefits of Open Science practices.16 23 Motivation for Open Science can be addressed by providing incentives, such as awarding funding to research-embedding open practices.57 Similarly, Open Science badges recognising open data, materials and pre-registration have been adopted by journals as a simple, low-cost scheme to reward these research behaviours.58 However, uptake of Open Science badges in physical activity journals is currently low and is rife for increased uptake in the field.
Strengths and limitations
A strength of this study is the implementation of a comprehensive and previously used approach to identify Open Science practices. Moreover, two researchers independently carried out the coding of Open Science practices, reducing the risk of human error and maximising reliability.59 A limitation is that the search and screening processes were conducted by a single author. However, unlike systematic reviews, we did not attempt to conduct a comprehensive search to identify all relevant research but to select a somewhat random subsample to analyse Open Science practices and inform specific recommendations for future research. In this regard, it is worth acknowledging that results are based on a relatively small sample of physical activity behaviour change reports, meaning findings may not be applicable to all physical activity research. Last, the assessment of Open Science practices was entirely dependent on what was described within evaluation reports. Direct requests to authors or additional wider searching of third-party registries such as Open Science Framework may have identified additional information.