Article Text
PDF
Abstract
Purpose A central aspect of physical activity and sedentary behaviour research is accurate exposure assessment in the context of disease outcomes. The primary objectives of this study were to evaluate the convergent validity and test–retest reliability of the ActiGraph GT3X+ and activPAL3 accelerometers.
Methods Participants from the Breast Cancer and Exercise Trial in Alberta (n=266) wore both devices concurrently during waking hours for 7 days. Summary measures of time (hours/day) for physical activity and sedentary behaviour were compared between devices using Student’s t-tests. Bland-Altman plots were used to assess or evaluate the mean differences and limits of agreement between monitors, and intraclass correlation coefficients (ICCs) were used to assess the test–retest reliability of two 7-day activity monitor administrations separated by 2 weeks (n=29).
Results When comparing the ActiGraph Vector Magnitude (VM), which incorporates all three axes of movement (x, y, z), and the Vertical Axis (VT), which detects movement on the vertical or y-axis only, with the activPAL3, all measures of physical activity were statistically significantly different. The difference in measured time in sedentary behaviour was not statistically significant different when comparing the activPAL3 and ActiGraph (VT) estimates (p=0.47) but was statistically significant different for activPAL3 compared with ActiGraph (VM) (p<0.001). ICCs were high and consistent for each method across all behaviours, ranging from 0.87 to 0.93, with the exception of moderate activity and moderate-to-vigorous activity by the ActiGraph (VT) at 0.66 and 0.67, respectively.
Conclusion Despite small mean differences and comparable recordings by both devices at the group level, the precision of estimates between methods was low with wide limits of agreement, suggesting these devices may not be used interchangeably for measuring physical activity and sedentary behaviour using common data reduction methods.
- Convergent validity
- reliability
- accelerometry
- women
- cross-sectional study
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
What are the new findings
The test–retest reliability (intraclass correlation coefficients) was high and consistent across all physical activity andsedentary behaviour estimates for the ActiGraph Vertical Axis (VT), Vector Magnitude (VM) and the activPAL3,indicating low behavioural variation between weekly recordings in a free-livingcondition.
Using common data reduction methods, the limits of agreement for all physical activity andsedentary behaviour estimates between ActiGraph VT, VM and activPAL3 werewide.
How might it impact on clinical practice in the near future
These results support previous research that recommends using the activPAL devicefor measuring sedentary behaviour.
Conversely, our results support the use of the ActiGraph device to assess physicalactivity in research studies assessing higher intensity activities.
For researchers who wish to use a single device, it is important to be aware of themethodological limitations of each instrument and the effects on outcomes of interest.
Introduction
At present, the methods for quantifying physical activity behaviour in epidemiological studies involve self-reported questionnaires or direct observations. These methods are used to obtain an estimate of either current or past activity patterns. An important aspect of physical activity and sedentary behaviour research involves quantifying these behaviours more accurately given some of the recognised limitations of self-report methods.1 This need stems from the importance of improving the internal and external validity of data collected on physical activity and health outcomes. Threats to validity in self-report measures include errors resulting from difficulty in estimating quantities of physical activity and social desirability bias, among others.2 By objectively measuring physical activity, several of these biases and threats to validity can be overcome, and a more accurate assessment of the association between physical activity and health outcomes can be achieved.
Sedentary behaviour is defined as activity performed while awake that is done in a seated or lying position and does not increase energy expenditure substantially, typically with a metabolic equivalent value (MET) of 1–1.5.1 3 4 Sedentary behaviour has been associated with several chronic and metabolic diseases, including cancer, diabetes and cardiovascular disease.5
Accelerometry has enabled the objective quantification of both physical activity and sedentary behaviour. The appropriate choice of accelerometer for epidemiological studies that seek to measure physical activity and sedentary behaviour objectively depends on the summary metrics that can be derived from a given monitor, as well as the validity and reliability of the estimates obtained. With many competing accelerometers available for use,6 it is unclear which accelerometer is optimally suited for estimation of physical activity and sedentary behaviour for the population under study.
The purpose of this study was to provide empirical evidence on the convergent validity and the behavioural variation or the test–retest reliability of two leading accelerometry devices that are currently in use: the ActiGraph GT3X Plus and the activPAL3. By comparing estimates using the typical data extraction methods of physical activity and sedentary behaviour from both devices worn concurrently in a large sample of postmenopausal women, an assessment of how well these devices agree can be made. This study provides additional value as it involves a large sample size with data collected from a free-living environment. In addition, recommendations regarding the most appropriate device or combination of devices for use in large-scale epidemiological studies are provided.
Methods
Participants
Participants in this study were from the Breast Cancer and Exercise Trial in Alberta (BETA), a two-armed randomised controlled exercise intervention trial conducted in Calgary and Edmonton between 2010 and 2013 that examined the effects of 12 months of moderate (150 min/week) versus high (300 min/week) volume of aerobic exercise on various hormonal and biological mechanisms that are hypothesised to be operative in the association between physical activity and breast cancer risk.7 The BETA study population consisted of 400 postmenopausal English-speaking women aged 50–74 years who were inactive at baseline. A detailed description of the sampling strategy and inclusion criteria of the BETA study participants is described elsewhere.7
Study design and data collection
This study was a cross-sectional analysis of accelerometer data collected at the 12-month time point in BETA when both accelerometers were worn simultaneously. The 12-month time point was used as this period included the most valid data for both devices and women were accustomed and comfortable with the protocol of wearing both monitors for 1 week. Participants were asked to wear the monitors for 7 days for all ‘waking hours’. The monitors were removed when they were sleeping and for any water-based activity. An activity monitor daily log was completed by each participant to record the time when the monitors were worn and what activities were done during ‘non-wear’ time. For the test–retest reliability study, a subset of 29 participants wore both monitors for two 7-day periods 2 weeks apart. The goal of this substudy was to assess the behavioural variation for each monitor administered based on the averages of summary measures from each time point as determined by each device’s common data extraction and reduction methods. This substudy was approved by the Conjoint Health Research Ethics Board of the University of Calgary. Informed consent was provided by all participants prior to participation.
Instrumentation
The ActiGraph GT3X+ device is an activity monitoring device worn around the waist that provides information on physical activity intensity in three dimensions using activity counts that reflect the duration and intensity of movement/activity for a given epoch. We used activity counts derived from the Vertical Axis (VT) method and the Vector Magnitude (VM) method,8 9 and also employed the inclinometer function to estimate sedentary time. We initialised the ActiGraph to capture raw acceleration information at a rate of 80 Hz and used the ActiLife software to determine wear time and to create 60 s epoch files with the low frequency extension selected. The ActiLife software was used to check that recordings had a minimum of 10 hours of wear time for at least 4 days. Graphic displays of activity data from the ActiLife software were also compared with the participants’ diary logs to ensure the minimum wear time criterion was met. All data from the ActiGraph device were collected when the device was worn at the waist.
Sedentary time was defined in two ways: (1) using VT activity counts below 100 counts per minute (CPM) and (2) using the ActiGraph’s inclinometer data. For this study, we grouped the estimate of sedentary time from the ActiGraph inclinometer under the VM method of measurement. The inclinometer function was a new addition to this ActiGraph device; thus, we included it in the VM method of data extraction to compare with the standard count cut-points to detect sedentary time for earlier ActiGraph devices without an inclinometer function. The VT variable for physical activity used the cut-points for light (100–1951 CPM) and moderate-to-vigorous activity (MVPA) (>1952), which were derived from walking and running activities.8 For VM, we used the cut-points for light (non-sedentary behaviour <2690 CPM) and MVPA (>2691), which were derived from treadmill walking and running.9
The activPAL3 device is worn on the front midline portion of the right thigh and adheres to the skin with the use of PAL stickies. It uses thigh position to differentiate between sedentary time (sitting or lying) from physically active time (standing/stepping). The activPAL3 collects data at a sampling rate of 20 Hz and it measures four quantities: time spent sitting or lying down, standing, and stepping, as well as step rate/cadence. The activPAL3 also estimates the energy cost of ambulatory activities using a prediction equation that employs stepping cadence and duration as the predictor variables (MET hours = (1.4 x duration [hours]) + (4–1.4) x (cadence [steps/min]/120) x duration). We used these data to estimate light (non-sedentary time <3 METs), moderate (3–5.9 METs) and vigorous activity (6+ METs).10 Data were downloaded using the activPAL3 software and we used the 15 s epoch file to calculate relevant summary variables. Similar to the ActiGraph device, the minimum wear time criteria were evaluated using the activPAL3 software, and graphical displays of data were compared with the participants’ diary logs to ensure completeness. The average time worn for the duration of the entire recording was used in the analysis. The activPAL has been validated in laboratory studies11 12 and in free-living studies13–16 in comparison with direct observation.
Statistical analysis
All estimates of physical activity and sedentary behaviour were compared using paired t-tests after assessing normality. The Bland-Altman method was used to assess agreement in time spent in each behavioural category.17 This method allows for the investigation of any systematic difference between measurements as well as the precision of the estimates, or the limits of agreement. Intraclass correlation coefficients (ICCs) were used to estimate the test–retest reliability of the 7-day activity monitor administrations separated by 2 weeks. For analysis, data from each monitor administration were averaged, so the ICCs represent the variability between administrations rather than variability from day to day. For this reliability substudy, non-wear time was manually removed using the diary log from each participant to obtain the most accurate estimate of true wearing time. This manual removal of non-wear time was feasible because of the relatively small sample size. These estimates were then expressed as a per cent of total wear time in order to correct for differences in wear time between the two recordings. Relative agreement was assessed using ICCs based on two-way mixed models without interaction terms. Analyses were performed using STATA V.12.0.
Results
Of the 400 randomised BETA participants, 306 (77%) had a simultaneous recording for both ActiGraph and activPAL3 at the 12-month time point. Of the 306 with a simultaneous recording from both devices, 266 (87%) had valid accelerometer data for both monitors. The reasons for invalid data included not obtaining the minimum wear time requirements of at least 4 days of wear for a minimum of 10 hours (n=27) and not being able to match the same valid wear days between devices (n=13).
A total of 266 postmenopausal women were included in the analysis to compare the ActiGraph GT3X+ and activPAL3. This study subsample at study entry had a mean age of 59.7 years (SD=5.0), ranging from 51 to 74 years (table 1). The participants were mostly Caucasian (89.1%), well educated with the majority (79.7%) having completed college, trade school or a university degree, were married or in common-law relationships (70.7%), and parous (88.7%). The majority of women in this sample were overweight, with a mean body mass index of 29.0 kg/m2 (SD=4.6). The mean waist circumference was 98.5 cm (SD=11.0), which exceeds the criterion cut-point for women with metabolic syndrome.18 This subset of 266 women did not differ in any demographic, anthropometric or lifestyle characteristics from the full study population included in BETA.19
Baseline characteristics of substudy participants in Breast Cancer and Exercise Trial in Alberta, 2010–2013
The devices were each worn for an average of nearly 15 hours per day, for which between 8 and 9 hours was spent in sedentary activity, and 6–7 hours in mainly light activity and a relatively small amount of time (<1 hour) in MVPA. Summary measures of the average time (hours/day) spent in physical activity and sedentary behaviour as detected by the ActiGraph GT3X+ and activPAL3 are shown in table 2.
ActiGraph GT3X+ VM, VT and activPAL3 characteristics (hours/day)
Summary measures comparison
ActiGraph VM and activPAL3
Statistically significant differences were found between the mean values obtained from the ActiGraph VM as compared with the activPAL3 for moderate-intensity physical activity (−0.28 hours/day), MVPA (−0.15 hours/day, p<0.001) and sedentary behaviour (0.47 hours/day, p<0.001) (figure 1). No statistically significant difference was found for light activity (−0.21 hours/day, p=0.07) (figure 1).
Comparison of the mean hours/day across activity type with the ActiGraph GT3X+ VT, VM and activPAL3. MVPA, moderate-to-vigorous activity; VM, Vector Magnitude; VT, Vertical Axis.
ActiGraph VT and activPAL3
Likewise, we observed statistically significant differences between the group mean duration estimates when comparing the ActiGraph VT and activPAL3 for all physical activity categories of light (0.70 hours/day), moderate (−0.61 hours/day) and MVPA (−0.53 hours/day) (all p values<0.001) (figure 1). No statistically significant difference was found for estimated time spent in sedentary behaviour (0.06) (p=0.47) (figure 1).
ActiGraph VT and ActiGraph VM
Statistically significant differences in reported mean estimates (mean difference ActiGraph VM minus VT hours per day) of physical activity and sedentary behaviour occurred across all physical activity intensities of light (−0.91 hours/day), moderate (0.33 hours/day) and MVPA (0.38 hours/day) (p<0.001). Estimated time in sedentary behaviour was also statistically significantly different between the two methods of VM and VT (0.53 hours/day) (p<0.001) (figure 1).
Agreement
ActiGraph VM and activPAL3
The level of agreement is reported as the mean difference in hours per day between ActiGraph VM and activPAL3 for estimates of sedentary behaviour and physical activity (95% limits of agreement). For sedentary time, the level of agreement was 0.46 (−3.3 to 4.2) hours per day when comparing the ActiGraph VM and activPAL3. For light-intensity activity, the estimated agreement was −0.2 (−3.9 to 3.4) hours per day and is shown as an example in figure 2. The estimated agreement for moderate activity was −0.3 (−1.0 to 0.5) hours per day and for MVPA was −0.2 (−0.9 to 0.6) hours per day. The mean difference for each comparable behaviour category was small. Despite this small difference, the limits of agreement are wide for each behaviour category, with the largest limits of agreement observed for sedentary behaviour and light activity, while the smallest was observed for moderate activity and MVPA.
Bland-Altman plot: light activity ActiGraph (AG (VM)) and activPAL3. VM, Vector Magnitude.
ActiGraph VT and activPAL3
For estimates of sedentary behaviour, the level of agreement was −0.1 (−2.7 to 2.6) hours per day. For light activity, the estimated agreement was 0.7 (−1.7 to 3.1) hours per day. The estimated agreement for moderate activity was −0.6 (−1.2 to −0.3) hours per day and for MVPA it was −0.5 (−1.0 to −0.02) hours per day. For each comparable behaviour, the mean difference or bias was small. However, with the exception of moderate and MVPA activity, the 95% limits of agreement were wide, spanning several hours.
ActiGraph VT and ActiGraph VM
The mean difference between methods for estimated time in sedentary behaviour was 0.5 (−2.8 to 3.8) hours per day. The largest mean difference between methods occurred for light-intensity activity estimated at −0.9 (−4.3 to 2.5) hours per day. The mean difference between VM and VT was small for moderate and MVPA activity at 0.33 hours per day (−0.2 to 0.85) and 0.38 hours per day (−0.14 to 0.90), respectively. The limits of agreement are wide for each behaviour category, with the largest limits of agreement observed for sedentary behaviour and light activity, while the smallest was observed for moderate activity and MVPA.
Behavioural variation (test–retest reliability)
Behavioural variation for two 7-day administrations, with a 2-week separation between administrations, was estimated using ICCs. The estimated reliability for sedentary behaviour was high and similar across all methods at 0.93 (0.86–0.97), 0.91 (0.81–0.96) and 0.93 (0.85–0.97) for VT, VM and activPAL3, respectively. The estimated reliability for light intensity activity was 0.91 (0.82–0.96), 0.87 (0.73–0.94) and 0.90 (0.79–0.95) for VT, VM and activPAL3, respectively. The estimated reliability for MVPA was 0.67 (0.31–0.84), 0.88 (0.75–0.94) and 0.87 (0.72–0.94) for methods of VT, VM and activPAL3, respectively (table 3).
Absolute reliability of physical activity and sedentary behaviour (hours/day) for the ActiGraph GT3X+ VM, VT and activPAL3
Discussion
The purpose of this study was to assess the convergent validity and test–retest reliability for estimates of physical activity and sedentary behaviour between the ActiGraph, including VT and VM methods, and the activPAL3 using data collected from a subset of participants in the BETA trial.7 When comparing estimates using the typical data reduction methods for physical activity between the ActiGraph GT3X+ VT, VM and activPAL3, statistically significant differences occurred for all intensities, with the exception of light activity between the ActiGraph VM and activPAL3. For estimated time spent in sedentary behaviour, no statistically significant difference was observed between the ActiGraph VT and the activPAL3, while a significant difference was found between the ActiGraph VM and activPAL3. When comparing the ActiGraph VT and VM methods of measurement for aggregate time in physical activity, statistically significant differences were found for each intensity of physical activity, including light, moderate and MVPA. The two different methods of detecting sedentary time by the ActiGraph, VT (<100 CPM) and VM (inclinometer), were statistically significantly different from each other. Bland-Altman plots assessing agreement between methods of measurement at the individual level showed that the limits of agreement for all physical activity and sedentary behaviour estimates between ActiGraph VT, VM and activPAL3 were wide. All comparisons of physical activity and sedentary behaviour between methods were significantly correlated. The test–retest reliability (ICCs) was high and consistent across all physical activity and sedentary behaviour estimates for the ActiGraph VT, VM and the activPAL3.
Similar to the study by Hart et al, the limits of agreement between methods for comparable behaviours are wide, indicating that the devices may not be used interchangeably.20 Martin and colleagues used Bland-Altman plots to assess agreement between ActiGraph and activPAL for per cent time in sedentary behaviour in preschool children.21 This study found that agreement at the individual level was poor, with mean difference (limits of agreement) of −4.3% (−14.0% to 5.0%).21 Martin and colleagues noted that at the group level, the estimated time in sedentary behaviour was similar despite the statistically significant difference (p<0.001).21 This finding is similar to the results in our study demonstrating that at the group level, the difference in time spent in sedentary behaviour between methods is small; however, agreement between methods at the individual level demonstrates that limits of agreement are too wide to use both devices interchangeably.
The test–retest reliability (ICCs) was high and consistent across all physical activity and sedentary behaviour estimates for the ActiGraph VT, VM and the activPAL3. These results indicate low behavioural variation between weekly recordings and suggest that participant recordings of at least 1 week are capturing habitual physical activity and sedentary behaviour patterns, at least in the short term.
Of interest is the finding that at the group level, no statistically significant difference occurred between the activPAL3 and the ActiGraph VT (<100 CPM) for time in sedentary behaviour. This finding may suggest that the ActiGraph device, using the common cut-point of <100 CPM, is accurately capturing sedentary time in a healthy population of older women if we consider that the activPAL device is most accurate and comparable with gold standards such as direct observation.11 13 14 16 21 Despite the similarity in group estimates of sedentary time between both devices, it must be noted that the limits of agreement are similarly wide between methods, and suggests that, at the individual level, these methods may not be used interchangeably. Previous studies, including one by Lyden and colleagues, aimed to assess sedentary behaviour using the ActiGraph GT3X and activPAL device compared with direct observation.14 This study found that the activPAL device was not statistically significantly different from direct observation for total time in sedentary behaviour, number of breaks in sedentary behaviour and break rate.14 In general, the AG100 CPM and AG150 CPM were not accurate in estimating time in sedentary behaviour compared with direct observation.14 Prior research comparing ActiGraph models with the activPAL for estimated time in sedentary behaviour also found that compared with direct observation, the activPAL device outperformed the ActiGraph and was more accurate in detecting sedentary behaviour.13 14 16 21 22 Although this study lacks an unequivocal gold standard such as direct observation, empirical evidence suggests that the activPAL has excellent measurement properties in comparison with direct observation for estimating time in sedentary behaviour and can be viewed as a strong criterion measure of sedentary behaviour. Our results suggest the ActiGraph device provides comparable estimates of sedentary time at the group level with the ActivPAL3; however, at the individual level, the ActiGraph is less precise.
It is important to take into consideration methodological shortcomings when comparing and evaluating two methods of measurement. This study was able to assess the convergent validity of the ActiGraph GT3X+ and the activPAL3 monitoring devices. Convergent validity results reported herein provide useful information as both devices are designed to measure the same parameters, and by estimating the agreement between monitors we can answer the questio n of whether or not these two devices can be used interchangeably. Furthermore, previous research has demonstrated that the activPAL device provides reliable and accurate estimates of sitting time compared with direct observation, and the activPAL is arguably becoming the gold standard for device-based sedentary behaviour measurement.11 13 14 16 21 22 It is important to note the difference between the ActiGraph VT and activPAL3 with respect to what activity is considered as sedentary time. The ActiGraph VT method, using count cut-points, assumes low energy expenditure but does not take into account body position. Thus, it is possible for the device to record standing activity as sedentary time if sufficiently low count data are produced. Conversely, the activPAL3 device takes into consideration postural allocation of sitting and lying as sedentary time while assuming low energy expenditure.
Another study limitation is the narrowness of the study sample and thus the restriction in the generalisability of the study findings. The study participants were enrolled in a randomised controlled exercise intervention trial, and the study inclusion criteria were very specific and tightly controlled. Consequently, the ICCs in the reliability analysis may be overestimated as women who are not involved in an intervention trial are likely to be more heterogeneous in their activity levels and produce greater variability in results. The generalisability of findings is limited to a similar study population, and comparisons of physical activity and sedentary behaviour estimates using these devices with other study samples could provide inaccurate results.
Conclusion
Despite the small median difference between devices for physical activity and sedentary behaviour at the group level, our results demonstrate that at the individual level and using these predetermined reduction methods, these devices may not be used interchangeably. Similarly, time in physical activity and sedentary behaviour may not be comparable using methods of VT and VM with the ActiGraph device. This study does not support a conclusion to use one device over the other for objectively measuring sedentary behaviour and physical activity in epidemiological studies. These results support previous research that recommends using the activPAL device for measuring sedentary behaviour as it is more accurate in capturing this domain of activity. Conversely, our results support the use of the ActiGraph device to assess physical activity in research studies assessing higher intensity activities. For researchers who wish to use a single device, it is important to be aware of the methodological limitations of each instrument and the effects on outcomes of interest.
References
Footnotes
Acknowledgements Initial study coordination was provided by Sana Fakih. Sarah MacLaughlin, Amy Speed-Andrews, Ciara Kallal, Natalie Ilkiw, Erica Roberts, Quinn Harris and Kristen Simone assisted with study coordination and data collection for BETA. Erin Korsbrek and Quinn Harris assisted with accelerometer data processing. Data cleaning and processing for the accelerometers was overseen by Dr Steven Szarka, with assistance by Megan Farris and Quinn Harris. Drs Lindsay McLaren and Patricia Doyle-Baker provided feedback on this project.
Contributors CF and KC designed and obtained funding for the Breast Cancer and Exercise Trial in Alberta. They oversaw the conduct of the trial, including data collection and analysis. CEM developed and oversaw the data collection of the accelerometry data and provided input on the analysis of these data. KAK assisted with the statistical analyses of these data. QW developed the statistical programs for the reduction of these data. TP designed, conducted, analysed and wrote this manuscript. All authors contributed to the writing of this manuscript and approved the final version.
Funding The Breast Cancer and Exercise Trial in Alberta was funded by the Alberta Cancer Foundation (Grant #24404). Dr Christine Friedenreich holds a Health Senior Scholar Award from Alberta Innovates-Health Solutions and the Alberta Cancer Foundation Weekend to End Women’s Cancers Breast Cancer Chair. Dr Kerry Courneya is supported by a Canada Research Chair in Physical Activity and Cancer. Ted Pfister received funding awarded to Dr Friedenreich for the Dean’s Prize in Publication and Mentorship from the Cumming School of Medicine, University of Calgary. The results of the present study do not constitute endorsement by ACSM. The results of the study are presented clearly, honestly and without fabrication, falsification or inappropriate data manipulation.
Competing interests None declared.
Ethics approval Conjoint Health Research Ethics Board of the University of Calgary.
Provenance and peer review Not commissioned; externally peer reviewed.