Article Text

Download PDFPDF

Validation methods for global and local positioning-based athlete monitoring systems in team sports: a scoping review
  1. Live Steinnes Luteberget1,
  2. Matthias Gilgien1
  1. Department of Physical Performance, Norwegian School of Sport Sciences, Oslo, Norway
  1. Correspondence to Live S Luteberget; livesl{at}nih.no

Abstract

Background/Objective Global navigation satellite systems (GNSS) and local positioning systems (LPS) are to date common tools to measure external training load in athletes. The aim of this scoping review was to map out and critically appraise the methods used to validate different GNSS and LPS used in team sports.

Method A total of 48 studies met the eligibility criteria and were included in the review. The reference systems applied in the validations, and the parameters investigated were extracted from the studies.

Results The results show a substantial range of reference systems used to validate GNSS and LPS and a substantial number of investigated parameters. The majority of the validation studies have employed relatively simple field-based research designs, with use of measure tape/known distance as reference measure for distance. Timing gates and radar guns were frequently used as reference system for average and peak speed. Fewer studies have used reference system that allow for validation of instantaneous dynamic position, such as infrared camera-based motion capture systems.

Conclusions Because most validation studies use simple and cost-effective reference systems which do not allow to quantify the exact path athletes travel and hence misjudge the true path length and speed, caution should be taken when interpreting the results of validation studies, especially when comparing results between studies. Studies validating instantaneous dynamic position-based measures is warranted, since they may have a wider application and enable comparisons both between studies and over time.

  • Assessing physiological demands of physical activity
  • validation
  • sports analysis in different types of sports
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

View Full Text

Statistics from Altmetric.com

INTRODUCTION

Objective analyses of physical training load in team sports can provide better understanding of the specific physical demands of a sport, the physical development of players over time, health and performance, and can help to improve training practices.1 2 Different methods for time-motion analysis, such as hand notation and video analysis, have been used to objectively assess training load for many decades. However, the time-consuming nature of such analysis has restricted its use.3 The development of wearable athlete monitoring systems has made objective athlete monitoring more available in team sports. Most wearable athlete monitoring systems consist of a global navigation satellite system (GNSS) for outdoor use or a local positioning system (LPS) for indoor use. GNSS and LPS systems provide meaningful position-based measures such as speed or path length for team sports. The use of GNSS-based and LPS-based athlete monitoring systems is now commonplace in team sports, and the number of research publications related to the application of these technologies in team sports is high and increasing exponentially (figure 1). Wearable athlete monitoring systems often also include inertial sensors, such as accelerometers and gyroscopes. These are typically used to measure acceleration and parameters based on acceleration. This article does not address inertial sensors but focuses on GNSS/LPS technology.

Figure 1

The increase in publications in the area of team sports and GNSS/LPS technology from 2000 to 2019. The figure was constructed using the ‘europepmc’ package in R. The whole code, including search words, can be found at the Open Science Framework (URL: https://osf.io/3h8qa/).

The large number of GNSS and LPS system applications in sport teams and research emphasise the importance of the question of whether these systems are sufficiently validated and can accurately measure what they are intended to measure. Good internal and external validity4 of data collection systems (eg, LPS or GNSS) applied in sports is important to allow meaningful analysis, enhance coaching and build trust between athletes, coaches and scientists in the application of such systems.5 One main reason why wearable athlete monitoring systems are applied in team sport is that they allow collection of data during real-life training and competition4 and hence substantially improve external validity compared to investigations in laboratory settings. The internal validity of a system is equally important. It reflects the ability to accurately measure what the system intends to measure.6 If the internal validity of a system is not adequate, training load can be overestimated or underestimated, and the application of such measurement systems may cause harm to athletes by the prescription of inadequate training, leading to decreased performance and/or increased health risks.5 7

Both GNSS and LPS are prone to measurement error, and there are many factors that can influence position validity. Calculation of the GNSS or LPS position of a wearable athlete monitoring system (receiver) is based on position and time information from satellites circulating around the earth (for GNSS) or local nodes mounted around the field of play (for LPS). Satellites and nodes emit an electromagnetic signal that is received by the receiver on the athlete. From these signals, there are several techniques that can be used to calculate instantaneous position, such as time-of-flight, time-difference-of-arrival, angle-of-arrival and received signal strength.8 GNSS use time-of-flight, while LPS vary between different systems in which technique they use. The main device-related factors that influence the validity of this kind of position measurement include antenna and board type, number of satellites/nodes used for position calculation, signal type used, processing method, measurement frequency and parameter calculation process.9 10 Since wearable tracking devices applied in sports should be small, light and user-friendly, the manufacturers of such devices optimise the trade-off between system performance, form factor, handling simplicity and cost. Due to these manufacturing compromises and the continuous system improvements in hardware and firmware, data processing and parameters, the validity of such systems needs to be investigated prior to use. To date, several validity studies have been published for GNSS,11 and to a lesser extent for newer LPS12 in team sports. The GNSS studies11 show a large range of standards (hereafter called reference systems) applied to validate wearable athlete monitoring systems and the parameters investigated.

In recognition of the importance validity has in match and training analysis in team sports, and the apparent range of validation methods applied in GNSS/LPS studies, this scoping review aims to present and critically appraise the methods used to validate the various GPS and LPS used in team sports.

METHOD

Review protocol

The protocol for this review is available at the Open Science Framework (URL: https://osf.io/3wn82/), where both the protocol and the full search strategy can be found (URL: https://osf.io/rmcgf/). This review was conducted and reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews.13

Eligibility criteria

Articles were eligible for inclusion in this review if they (1) included investigation of validity/accuracy for GNSS or LPS and (2) were aiming to investigate this in relation to team sports. Articles were excluded if they were (1) published in a non-English language or (2) only available in conference abstract or conference proceedings format. Reviews or other studies with no primary data were not included in this scoping review.

Search strategy

A systematic electronic database search was conducted in SPORTDiscuss and PubMed for all published manuscripts prior to the search date (15.09.19). The search strategy included the following terms (and variations of these terms): ‘Global Positioning System’ OR ‘Global Navigation Satellite System’ OR ‘Local Positioning System’ AND ‘Validity’ OR ‘Accuracy’ AND ‘Team Sports’. The full search strategy can be found at the Open Science Framework (URL: https://osf.io/rmcgf/). No filters or limitations were imposed during the search.

Study selection

Search results were exported to a reference manager library (Endnote, X9.2), where duplicates were removed. The citations were then uploaded to the systematic review software DistillerSR (Evidence Partners, Ottawa, Canada). Titles and abstracts of the citations were screened for eligibility independently by two reviewers. Full texts of potentially eligible articles were retrieved before a final assessment was completed independently by the same two reviewers. Any discrepancies between reviewer eligibility assessments were resolved through discussion with a third reviewer. All three reviewers were familiar with the topic of the review.

Data extraction

GNSS/LPS specifications (brand, model and sampling frequency), sporting tasks assessed, reference system used for the validation, and parameters investigated were extracted from the included studies. Tasks were classified into four different categories: linear (straight line) tasks, non-linear tasks, team sport circuits and game-like situations (eg, small-side games). The type of reference system used to assess validity was extracted as stated in the studies. The parameters for time, averaged static position, instantaneous dynamic position, distance travelled, average speed, peak speed, instantaneous speed, average acceleration, peak acceleration and instantaneous acceleration were extracted. Other parameters, such as metabolic power or time to cover distance, were categorised as ‘other’. Data extraction was performed by two independent reviewers.

RESULTS

The database search identified 454 relevant records. Duplicates (n=76) were removed, so 378 titles and abstracts were reviewed. A total of 48 studies met the eligibility criteria and were included in the review.12 14–60 An overview of the search and selection process is presented in figure 2.

Figure 2

Study selection flow chart.

The studies investigated from one to five parameters each. Distance was the most frequently investigated parameter (34 articles), followed by average and peak speed. Fewer studies investigated dynamic position, or instantaneous speed and acceleration (figure 3).

Figure 3

Proportion of GNSS/LPS studies investigating different parameters.

Five different reference systems were used to investigate the validity of distance, where tape measure/known distance constituted the most frequently used reference systems. For validation of speed the reference systems applied were timing gates, radar gun and infrared camera-based motion capture systems. For the validation of acceleration only infrared camera systems were used. A summary of the results is given in table 1, while a full documentation of the different reference systems used and parameters assessed is shown in table 2 (LPS) and table 3 (GNSS).

Table 1

Overview of different reference systems used to validate the most common performance and training load parameters

Table 2

Overview of included studies investigating validity of LPS

A variety of different tasks are used to investigate the validity and accuracy of GNSS/LPS. Linear tasks were the most frequently used (tables 2–3) and were included in most studies. Different circuits and courses imitating team sports movements were also frequently used. Game-like situations were only used in three of the 48 included studies (tables 2–3).

Table 3

Overview of included studies investigating validity of GNSS

DISCUSSION

This study provides an overview of the published, peer reviewed studies investigating the validity of GNSS and LPS in team sports. Since the first validation study on GNSS in team sports was published in 2006,27 the number of validation studies has steadily increased in this field. It seems that the increasing number of validation studies is required, since the number of manufacturers and types of GNSS/LPS-devices, and with these the variety of hardware and firmware, sampling rates and data-processing methods, have increased. In total, the validity of at least 23 GNSS and six LPS models—from 17 different manufacturers—for team sports applications have been investigated in the literature.

The results show a substantial range of reference systems employed to validate GNSS and LPS, and a substantial number of parameters that were investigated. Most of the validation studies have employed relatively simple field-based research designs, using a tape measure/known distance as the reference system for distance. Timing gates and radar guns were frequently used as reference systems for average and peak speed. Fewer studies have used reference systems that allow for validation of instantaneous dynamic position, such as infrared camera-based motion capture systems.

Distance travelled and peak and average speed were the most frequently investigated parameters. The high number of studies investigating these parameters is justified by their frequent use in time-motion analyses in team sports.12 32 33 Only a few studies have investigated the validity of instantaneous dynamic position, which may be due to the unavailability of appropriate reference systems, such as infrared camera-based motion capture systems. However, some studies did not provide instantaneous dynamic position, even though the reference system applied could have provided this information.25 26 55 We believe that insight into the validity of instantaneous dynamic position could be beneficial for several reasons. First, other parameters (such as distance) are integrations or derivatives of instantaneous dynamic position and hence, deviations in position measurement are propagated to these parameters and potentially amplified by data processing, such as filtering and parameter calculation methodology. Such data processing steps will likely deviate between devices and manufacturers. Thus, appropriate validations of a system’s instantaneous dynamic position would allow comparison of the system’s ability to measure the basic parameter (position) and allow pinpointing of a) the error caused by the basic parameter (position) measurement and b) the manufacturer’s data processing. Second, parameters such as distance or speed can be affected by firmware update-related changes in the manufacturer’s data processing (typically parameter calculation and filtering). Hence, altered firmware may cause differences in the propagation parameters such as distance and speed compared with earlier firmware versions, even though the measurement of the basic parameter (position) may remain unchanged. It is likely that system improvements more often affect data processing (parameter calculation and filtering) than the basic measurement (instantaneous dynamic position), as the GNSS and LPS are often provided by a third-party manufacturer. Therefore, position could be used as a more stable long-term parameter for determining the basic validity of a system. Third, GNSS/LPS data are also used for tactical analyses, including parameters such as mean position over time and dynamic distances between players,61 62 which are based on position. Therefore, it is important that studies also investigate the validity of instantaneous position. We therefore suggest that the validity of instantaneous dynamic position should be included in validation studies, as it may have a wider application and can in the long run be both time and cost saving due to its more long-term stability across firmware versions.

Some studies lacking an appropriate reference system for instantaneous dynamic position have investigated time-averaged static positions.29 52 54 Two studies have measured positions as reference points,52 54 while one study29 applied the average measured position of the receiver as a reference. These two validation methods for static position are inherently different and may elicit vastly different results. The average position measurement obtained using the same device as the one to be validated provides only random error and cannot measure the systematic deviance from the true location. Thus, if the true static position is unknown, the relative position difference should be stated as a precision measure, not an accuracy or validity measure.

Several validation studies have used premeasured distances as reference systems for distance and average speed.20–24 28 29 31–36 42 44 46 47 51 54 60 This is a simple and cost-effective way to investigate the validity of tracking systems. However, the method is not an ideal reference system, as it is not possible to quantify the exact path travelled by the athlete as long as the athlete’s true path is not tracked instantaneously. During human locomotor tasks the individual and thus the device will seldom follow a straight line between two points. This could affect the outcome of validation studies, as is pointed out by some authors.24 Thus, smaller or larger deviations in the athlete’s position may go undetected and can lead to an underestimation or overestimation of the accuracy of the investigated system. To avoid this problem, the use of reference systems that measure the true instantaneous trajectory of the athlete’s device, such as infrared camera-based motion capture systems,12 25 26 40 45 55 59 video-based tracking,10 63–65 or, previously validated high-end GNSS devices,66 is warranted. Such reference systems also make it possible to investigate more complex tasks, such as game-like situations, which are inherently the most specific conditions to test the systems in.

Timing gates are also easy to apply and are often used as the reference system for mean speed, and in some cases peak speed15 24 33 34 and instantaneous speed16 . However, timing gates only determine mean speed in the sections between gates. Mean speed provides only limited insight in team sport applications, since it does not contribute much to the understanding of team sports, where speed constantly fluctuates as a function of the acceleration and deceleration of the athlete. Team sport analysis systems often sort speed data into ranges (speed zones) and express these as a function of time or distance as a comprehensive metric for the ‘distribution of intensity’ of the athletes’ physical load.67 Even though instantaneous speed measurements are commonly used to categorise speed as a function of time or distance, most validation studies only investigate the validity of mean speed over time. This is a serious shortcoming, since mean speed over time may not allow conclusions to be reached on the described distribution of intensity, which is based on instantaneous speed.

Some studies include the validity of peak speed; however, only a few studies have looked at the instantaneous speed over the range of a whole task. Radar guns were used in several studies to assess peak and instantaneous speed.19–21 35 37 43 49 53 56 58 The validity of radar guns during non-straight-line running is currently unknown, and they are thus used only in straight-line sprints in the current literature. Hence, a radar gun is not a suitable reference system for team sports motion, since most team sports involve mostly non-straight line motion. Reference systems such as infrared camera-based motion capture systems, video-based tracking, or previously validated high-end GNSS devices are warranted.

CONCLUSION

The most frequently investigated parameter in GNSS and LPS validity studies was distance travelled, followed by average and peak speed. Tape measure/known distance was the most frequent reference system applied. Few studies have investigated instantaneous parameters, such as instantaneous dynamic position or instantaneous speed. We discovered a large range of reference systems and methods employed to validate wearable athlete monitoring systems; thus, the appropriateness of the employed reference systems may vary, and caution should be applied when interpreting the results of validation studies, especially when comparing results between studies. More studies investigating instantaneous dynamic position may have a wider application and enable comparisons both between studies and over time.

Summary box

What is already known?

  • The use of GNS-based or LPS-based athlete-tracking devices is exponentially increasing.

  • Validation of GNSS/LPS is important to allow meaningful analysis in sports.

What are the new findings?

  • Known distance and timing gates are the most common reference systems in GNSS/LPS validation studies.

  • Few studies have investigated instantaneous dynamic position, the raw measurements of GNSS and LPS.

  • Caution should be applied when interpreting and comparing results of different validation studies due to the large variations in current validation methods.

Acknowledgments

We would like to thank Petter Jølstad for being involved in the article selection process.

REFERENCES

View Abstract

Footnotes

  • Twitter Live Steinnes Luteberget @livesl

  • Contributors LSL and MG contributed to the design and implementation of the research, to the analysis of the results and to the writing of the manuscript. LSL drafted the first version of the manuscript. Both authors contributed to the intellectual content of the study, manuscript writing and approved the final version of this article.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Ethics approval Not applicable.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.