Methodological quality and risk of bias of included studies assessed with the Quality Appraisal of Reliability Studies*
| Study | Item 1 | Item 2 | Item 3 | Item 4 | Item 5 | Item 6 | Item 7 | Item 8 | Item 9 | Item 10 | Item 11 |
| Ageberg et al13 | Y | Y | Y | NA | U | U | U | NA | Y | Y | Y |
| Barker-Davis et al56 | Y | Y | Y | U | U | NA | U | N | Y | Y | Y |
| Chmielewski et al14 | Y | Y | U | U | NA | U | U | Y | Y | Y | Y |
| Cornell et al66 | Y | Y | NA | U | NA | U | U | U | Y | Y | N |
| Crossley et al19 | Y | Y | U | U | Y | U | U | Y | Y | Y | N |
| Di Mattia et al20 | Y | Y | U | NA | U | U | U | NA | Y | Y | N |
| Edmondston et al59 | Y | Y | U | NA | U | U | U | Y | Y | Y | N |
| Friedrich et al61 | Y | Y | Y | NA | NA | NA | Y | U | U | Y | Y |
| Frohm et al15 | Y | Y | Y | NA | NA | Y | U | NA | Y | Y | Y |
| Gianola et al57 | Y | Y | Y | U | U | NA | U | U | Y | Y | Y |
| Harris-Hayes et al16 | Y | Y | Y | U | Y | U | U | Y | Y | Y | Y |
| Herman et al21 | Y | Y | U | NA | Y | U | U | NA | Y | Y | Y |
| Junge et al17 | Y | Y | U | NA | NA | U | U | NA | Y | Y | Y |
| Kaukinen et al58 | Y | Y | Y | Y | NA | Y | U | Y | Y | Y | Y |
| Kennedy et al23 | Y | Y | U | U | NA | U | U | U | Y | Y | Y |
| Lenzlinger-Asprion et al51 | Y | Y | Y | U | NA | Y | Y | Y | Y | Y | Y |
| McKeown et al24 | Y | Y | U | U | NA | U | U | NA | Y | Y | N |
| Nae et al30 | Y | Y | Y | NA | NA | U | U | U | Y | Y | Y |
| Park et al25 | Y | Y | Y | NA | NA | U | U | Y | Y | Y | Y |
| Piva et al26 | Y | Y | Y | NA | NA | U | U | Y | U | Y | Y |
| Poulsen et al52 | Y | Y | Y | Y | U | U | U | U | Y | Y | Y |
| Rabin et al64 | Y | Y | Y | NA | NA | U | U | NA | Y | Y | Y |
| Rabin et al65 | Y | Y | Y | NA | NA | U | U | NA | Y | Y | Y |
| Räisänen et al53 | Y | Y | U | Y | U | U | U | Y | Y | Y | N |
| Stensrud et al27 | Y | Y | NA | U | U | U | U | U | U | Y | Y |
| Teyhen et al60 | Y | Y | Y | NA | NA | U | U | Y | Y | Y | Y |
| Van Mastrigt et al63 | Y | Y | Y | NA | U | Y | U | Y | Y | U | Y |
| Weeks et al22 | Y | Y | Y | Y | Y | Y | U | Y | Y | Y | N |
| Weir et al54 | Y | Y | U | U | NA | U | U | Y | Y | Y | Y |
| Whatman et al18 | Y | Y | U | U | Y | U | U | Y | Y | Y | Y |
| Örtqvist et al55 | Y | Y | Y | U | NA | U | U | Y | U | Y | Y |
*Assesses study quality based on 11 items. Items: 1. Was the test evaluated in a sample of subjects who were representative of those to whom the authors intended the results to be applied? 2. Was the test performed by raters who were representative of those to whom the authors intended the results to be applied? 3. Were raters blinded to the findings of other raters during the study? 4. Were raters blinded to their own prior findings of the test under evaluation? 5. Were raters blinded to the results of the accepted reference standard or the disease status for the target disorder (or variable) being evaluated? 6. Were raters blinded to clinical information that was not intended to be provided as part of the testing procedure or study design? 7. Were raters blinded to additional cues that were not part of the test? 8. Was the order of examination varied? 9. Was the stability (or theoretical stability) of the variable being measured taken into account when determining the suitability of the time-interval among repeated measures? 10. Was the test applied correctly and interpreted appropriately? 11. Were appropriate statistical measures of agreement used?
N, no; NA, not applicable; U, unclear; Y, yes (marked in bold).