Abstract
Few studies have examined the validity of actigraphy in school-aged children. The objective of this study was to examine the validity of a commonly used actigraph compared to polysomnography (PSG) in a sample of children age 5 to 12 y born prematurely, sleeping in their natural home environment.
148 children born preterm (85 boys and 63 girls), ages 5–12 y (mean = 9.3 y, standard deviation = 2.0) wore the Philips Respironics Actiwatch-2 for 1 night concurrently with comprehensive, ambulatory PSG in the child's home. Sleep outcome variables were sleep onset latency, total sleep time (TST), and sleep efficiency. Epoch-by-epoch comparisons were used to determine sensitivity, specificity, and accuracy. Secondary analyses examined differences between children with no sleep issues, obstructive sleep apnea syndrome, and periodic limb movements in sleep (PLMS).
Actigraphy significantly underestimated TST (30 min) and sleep efficiency (5%). Actigraphy underestimated or overestimated sleep onset latency by at least 10 min for a third of the children. Sensitivity and accuracy were good at 0.88 and 0.84, respectively, whereas specificity was lower at 0.46. Differences between actigraphy and PSG for TST and sleep efficiency were greatest for children with PLMS.
This study adds to the small existing literature demonstrating the validity of actigraphy in middle childhood. Although actigraphy shows good sensitivity (ability to detect sleep), specificity (ability to detect wake) is poor in this age group. Further, the results highlight the importance of considering whether a child has PLMS when interpreting actigraphic data, as well as the difficulties in accurately capturing sleep onset latency with actigraphy.
The use of actigraphy to measure pediatric sleep-wake patterns is increasing, yet there remains little data about the validity of these devices in middle childhood. This study provides additional support for the use of actigraphy in pediatric research, yet also highlights the special considerations when interpreting the data provided by actigraphy. Future research is needed to not only further validate actigraphic devices, but also to improve the algorithms to improve the ability to detect wake after sleep onset in pediatric populations.
Introduction
Actigraphy is a noninvasive method of estimating sleep patterns through the monitoring of movement. A small, watch-sized device is worn on the wrist and a built-in accelerometer collects data on gross motor activity. These data are subsequently translated to epochs of wake or sleep using a device-specific algorithm.
Actigraphy has been shown to be useful in the assessment of sleep in both normal healthy individuals as well as in those with specific disorders such as circadian rhythm disturbances and obstructive sleep apnea syndrome (OSAS).1 Compared to sleep questionnaires and sleep diaries, actigraphy is more objective in its evaluation of total sleep time and sleep patterns, and avoids recall bias.2,3 The benefits of actigraphy also include the ability to do continuous recording over extended periods, as well as being less costly and less intrusive than overnight polysomnography (PSG). The convenience of being able to perform actigraphy in the comfort of one's home also makes it particularly attractive for use with children.
Despite the many uses and benefits of actigraphy, one of the major limitations is its inability to clearly distinguish between wakefulness and movements in sleep.2 Amidst the growing interest in actigraphy research in the past 2 decades, there remains a lack of large validation studies in children, with at best one validation study per device for each developmental age group. As children tend to move around in sleep more than adults, more studies are needed to understand the validity of actigraphy in children, as well as the strengths and limitations of these devices in children both with and without sleep disorders.4,5 The device examined in the current study, the Actiwatch-2 (Philips Respironics, Bend, OR), has only been validated in one study of school-aged children (n = 50) referred for an overnight in-hospital sleep study.6 As this previous validation study was conducted in a clinical sleep laboratory with a clinically referred population, it may not have been representative of nonclinically referred children. Thus, the purpose of the current study was to examine the validity of the Actiwatch-2 when directly compared to overnight PSG conducted in the child's home according to their usual sleep-wake schedules for a large sample of 5- to 12-y-old children born prematurely.
Methods
Study Group and Procedure
The study sample included 201 children (aged 5 to 12 y) who were participating in a larger study examining the long-term effects of caffeine therapy for apnea of prematurity on sleep.7 All children were born premature, weighed 500–1,250 g at birth, did not have major congenital anomalies or syndromes, and were randomized to either caffeine or placebo arms of the initial study, receiving this therapy for 6 w in the neonatal period.8 Follow-up evaluations with these children occurred 5 to 12 y after the intervention and 148 of the 201 children had a comprehensive ambulatory home PSG performed concurrently with actigraphy.9 No differences were found between the caffeine and placebo groups in terms of sleep at the follow-up evaluation,7 thus the data for the two groups were combined. Current caffeine intake was reported to be low for children in both groups, averaging one drink per day for 26% for the caffeine group and 23.5% of the placebo group.7 The parent study was approved by each site's human subjects review board, and informed consent was obtained for all the subjects. The methodology and results of the primary study have been published previously.7
Polysomnography
A single-night, unattended, comprehensive ambulatory PSG was performed for all study subjects. Detailed methodology is published,9 but in brief, sleep technologists went to the children's homes to set up and remove the monitoring leads as close to their usual bedtimes and rise times as possible. Recorded parameters included: electroencephalography (F3-M2, F4-M1, C3-M2, C4-M1, O1-M2, O2-M1); left and right electrooculogram; sub-mental and tibial electromyograms; electrocardiogram; oronasal airflow via thermistor; nasal pressure with pressure transducer; rib cage and abdominal wall motion via respiratory inductance plethysmography, and oxyhemoglobin saturation with pulse waveform (Siesta 802, Compumedics, Charlotte, NC).
The studies were scored and interpreted by one sleep technologist (JT) and one pediatric sleep medicine specialist (CLM), in accordance with the American Academy of Sleep Medicine pediatric scoring criteria.10 The sleep period was scored from “lights out” to “lights on.” Children were considered to have OSAS if the obstructive apnea-hypopnea index was > 2/h7 and to have periodic limb movements in sleep (PLMS) if the periodic limb movement index was > 5/h.7,11
Actigraphy
Data Analysis
For each patient, epoch-by-epoch matching of their PSG and actigraphy was done. The PSG “lights off” and “lights on” times were applied to the actigraphy as the “start” and “end” times of the sleep period. The 1-min actigraphy epochs had to be matched with two 30-sec PSG epochs, so 1-min (two 30-sec epochs) of the PSG recording was collectively scored as “wake” if either of the 30-sec epochs was “wake.” If both consecutive 30-sec PSG epochs were scored as “sleep,” the 1-min PSG recording was scored as “sleep.”1,6,13
PSG sleep onset was defined as the time of the first epoch of sleep scored on the PSG. Actigraphic sleep onset was defined as the first minute of a consecutive 10 min of scored sleep, with 1 min of activity allowed within the 10 min. The following definitions for the three outcome variables were used: total sleep time (TST: number of minutes scored as sleep between lights off and lights on), sleep onset latency (SOL: number of minutes from lights off until sleep onset, as defined above), and sleep efficiency (SE: proportion of TST against the total number of minutes between lights off and lights on, expressed as a percentage).
Analyses were conducted using SPSS 22.0 (IBM Corp., Armonk, NY). Primary analyses included repeated-measures analyses of covariance (controlling for OSAS and PLMS) to evaluate differences between PSG and actigraphy for the three sleep outcome variables (TST, SOL, and SE). Epoch-by-epoch comparisons between actigraphy and PSG were used to determine sensitivity (i.e., ability of actigraph to detect true sleep), specificity (i.e., ability of actigraph to detect true wake), and accuracy (i.e., ability of actigraph to detect both sleep and wake).6,14
Secondary analyses used the Wilcoxin signed-rank test to examine differences between PSG and actigraphy for the three sleep outcome variables (TST, SOL, SE) separately for children with no sleep disorder, children with OSAS, and children with PLMS. A Bonferroni correction was used to correct for multiple comparisons, with P < 0.017 considered significant.
Results
Study Group
Participants included 85 boys and 63 girls, with a mean age of 9.3 y (standard deviation = 2.0 y). Maternal race was 83.8% White, 5.4% Asian, 10.1% Black, and 0.7% Other; 76 participants were from the Canadian sites and 72 participants were from the Australian sites. Based on the PSG results, 10 participants had OSAS, 17 had PLMS, 2 had both OSAS and PLMS, and 119 had neither OSAS or PLMS. Because the secondary analyses includes group comparisons (no sleep disorders, OSAS, PLMS), the two children with both sleep disorders were excluded, resulting in a final sample of 146 children.
Actigraphy Versus PSG
Controlling for sleep disorders, a significant difference was found for all three sleep outcome variables (Table 1,Figure 1). Specifically, the Actiwatch-2 underestimated TST by 30 min and overestimated SOL by 2 min, resulting in an underestimation of SE by 5%. Sensitivity to detect sleep was 0.88, specificity to detect wake was 0.46, and accuracy was 0.84. Notably, the difference in SOL between actigraphy and PSG ranged from −98 to 69 min. Actigraphy overestimated SOL by 10 or more minutes for 18% (n = 26) of the children, and underestimated SOL by 10 or more minutes for 16% (n = 24) of the children.
Differences between actigraphy and polysomnography, controlling for sleep disorders.
Differences between actigraphy and polysomnography, controlling for sleep disorders.
Scatterplots comparing actigraphy vs. polysomnography for (A) total sleep time, (B) sleep onset latency, and (C) sleep efficiency. An identity line highlights perfect agreement between the two measures. Blue dots are participants with OSAS, red dots are participants with PLMS, and white dots are participants with no sleep disorder. Dots under the line show where actigraphy has underestimated the sleep variable, while dots above the line show where actigraphy has overestimated the sleep variable.
Sleep Disorders
For children with no sleep disorder and those with PLMS, actigraphy significantly underestimated TST (difference in median values 26 and 56 min, respectively) and underestimated sleep efficiency (5% and 6%, respectively) (Table 2,Figure 2). No significant difference was found for SOL for children with no sleep disorder and those with PLMS. No significant differences were found between PSG and actigraphy for children with OSAS. No differences were found between sleep disorder groups for sensitivity, specificity, or accuracy.
Differences between actigraphy and polysomnography for total sleep time, sleep onset latency, and sleep efficiency by sleep disorder group.
Differences between actigraphy and polysomnography for total sleep time, sleep onset latency, and sleep efficiency by sleep disorder group.
Boxplot graphs showing the difference between actigraphy and polysomnography for the three sleep groups (no sleep disorder, OSAS, and PLMS) for (A) total sleep time, (B) sleep onset latency, and (C) sleep efficiency. The horizontal line shows perfect agreement (difference of 0), with boxplots under the line showing where actigraphy has underestimated the sleep variable, while boxplot above the line showing where actigraphy has overestimated the sleep variable.
Discussion
This study contributes to the sparse literature on the validity of actigraphy in pediatrics by demonstrating that actigraphy provides a valid measure of TST in middle childhood in a large sample of youth. To our knowledge there have been only four previous studies that have examined the validity of actigraphy compared to PSG in this age group. Only one of these included youth who were not referred for clinical PSG, but the sample was only 16 youth ages 10–16 y and used the Ambulatory-Monitoring Inc. (Ardsley, NY) AMA-32 device (Sadeh algorithm).13 The other three studies included a broader age range of youth (1–12 y using the Actiwatch AW-64 (Mini Mitter [evaluating multiple thresholds], Bend, OR),15 2–18 y using the Actiwatch AW-64 and the automatic threshold,16 and 3–18 y using the Actiwatch-2 evaluating multiple thresholds6), but only one of these reported separate results for middle childhood,6 and none compared actigraphy to ambulatory PSG with the children sleeping in their natural home environment based on their normal schedules. Further, this study provides valuable information on the utility of actigraphy in children with OSAS and PLMS.
In general, sensitivity and accuracy were similar to previous reports,6,13,15,16 demonstrating the strength of actigraphy to estimate sleep in youth. Although we found low specificity (or ability to estimate wake during the sleep period), our results were similar to three studies with clinically referred samples.6,15,16 Together these studies provide a reminder that actigraphy should be used with caution,2,4 and with the understanding that actigraphy has both strengths (multiday assessment that does not rely solely on parent report) and limitations (overestimation of wake during the sleep period).
Although no significant difference was found between the average SOL for PSG and actigraphy, the wide individual difference in terms of SOLs were notable, ranging from actigraphy underestimating SOL by 98 min to actigraphy overestimating SOL by 69 min. With a third of this sample having a SOL that was either overestimated or underestimated by at least 10 min, these results once again highlight the significant limitations of using actigraphy as a measure of SOL.2–4 Further, we had a clearly identified “bedtime” in the form of the PSG lights-out time. When actigraphy is used in the field, a clinician or researcher must rely on a sleep diary and/or event marker to designate the “bedtime,” which can further call into question the accuracy of the SOL.
As mentioned, one of the most notable strengths of this study is the large sample of nonclinically referred children ages 5 to 12 y. Although these children were recruited from the neonatal unit as infants, most of the participants did not have a sleep disorder, providing additional normative data to the existing literature. Further, this study explored differences in validity and sleep outcomes between children with and without OSAS and PLMS, highlighting the role of actigraphy for both children with and without sleep issues. Finally, this study compared actigraphy to comprehensive ambulatory home PSG conducted in the child's regular sleep environment.
Although only a small number of the children in this study had a sleep disorder, differences in clinical outcomes between PSG and actigraphy were significantly different for children without sleep issues and those with PLMS. However, it is not possible to draw conclusions about the role of actigraphy for children with OSAS because these analyses were likely underpowered to detect significant differences. However, similar to the other groups, actigraphy underestimated TST and SE for children with OSAS. Differences in actigraphically measured sleep outcomes between children with and without sleep disorders is important for both clinicians and researchers to consider when using actigraphy. In particular, the differences between PSG and actigraphy were greatest for children with PLMS, with actigraphy underestimating TST by 56 min. This highlights that the increased movement that results in PLMS is detected by actigraphy, even when worn on the nondominant wrist. However, we only had 18 youth with PLMS, thus further research is needed in this area. In general, our findings suggest that when actigraphy is used in a research study or clinical setting, the interpretation of data should be considered in the context of whether the child has a sleep disorder.
Future research should address the limitations of the current study. First, as is common in actigraphy validation studies, we included only 1 night of assessment. Although this single night was conducted in the home environment, multiple nights of measurement provide more accurate estimates of validity for actigraphy.17 For example, as with previous validation studies we did not truly capture “sleep offset” because our wake time was determined by the “lights on” time of PSG rather than the child's spontaneous waking. Thus, multiple consecutive nights of PSG assessment would allow for a more naturalistic ability to compare sleep onset and sleep offset times. Second, although our data provide support for the validity of actigraphy in school-aged children, additional validation studies using this device are needed in preschool children and adolescents. Third, our sample included only ex-preterm infants, which may limit the generalizability of our findings. Fourth, we only included one age group, one brand of actigraph device, examined one sensitivity threshold (although this was selected based on previous validation studies in this age group with the device), and utilized the manufacturer-provided software. Additional validation studies of other devices and age groups are needed, as well as studies that examine different algorithms. Finally, collapsing the two 30-sec PSG epochs into a single 1-min epoch may have influenced the specificity results.
In addition to these limitations, it is important for researchers and clinicians to understand that for scoring purposes a daily diary/log and/or event marker use is necessary in order to accurately identify sleep periods. Further, standard terminology should be adopted to ensure that results are comparable across studies. Two recent papers provide additional information and recommendations.4,18
In conclusion, this study provides a significant contribution to the literature by providing data on the validity of a commonly used actigraphy device for children ages 5 to 12 y. With the increased number of recommended clinical uses for actigraphy in the International Classification of Sleep Disorders, Third Edition,11 and the rapid rise in studies utilizing actigraphy in pediatric populations,4 it is important to have multiple validation studies like this one for each device that is currently being used.
Comments