Variables | Coder questions | Response options |
Article characteristics | ||
Coder instructions: Check the institutional affiliation of the corresponding author. If there are multiple corresponding authors, choose the first. If no corresponding author is identified, choose the first. If there are multiple affiliations for the selected author, choose the first. | ||
Country | Which country is the corresponding author based in according to their affiliated? | (List countries) / Unclear / Other |
Pre-registration | ||
Definitions: ‘Pre-registration’ refers to the specification of important aspects of the study (typically hypotheses, methods and/or analysis plan) prior to commencement of the study. | ||
Coder instructions: Check specific sections in the paper where these files might be located, for example, supplementary materials, appendices, author notes, methods and results sections. Search for ‘registration’. | ||
Pre-registration statement | Does the article state whether or not the study (or some aspect of the study) was pre-registered? | Yes – the statement says that there was a pre-registration / Yes – the statement says that there was NO pre-registration / No – there is no pre-registration statement / Other* |
Pre-registration method | Where does the article indicate the pre-registration is located? | Open Science Framework / AsPredicted / ClinicalTrials.gov / AEA Trial Registry / EGAP Registry / Registered Report / Other* |
Pre-registration accessible | Can you access and open the pre-registration | Yes / No / Other* |
Pre-registration content | What aspects of the study appear to be pre-registered? (select all that apply) | Hypotheses Methods Analysis Plan Other* |
Protocol sharing | ||
Definition: ‘Protocol’ refers to a document containing details about the study design, methods and analysis plan. It may or may not be pre-registered. | ||
Coder instructions: Search the article for the phrase ‘protocol’ and assess whether a link is provided to a protocol document. | ||
Protocol availability | Does the article link to an accessible protocol? | Yes / No / Other* |
Protocol content | What aspects of the study appear to be included in the protocol? (select all that apply) | Hypotheses Methods Analysis Plan Other* |
Data sharing | ||
Definitions: ‘Data’ refers to recorded information that supports the analyses reported in the article. A ‘data availability statement’ can be as simple as a url link to a data file, or as complex as a written explanation as to why data cannot be shared. | ||
Coder instructions: Check the article for a data availability statement/link. They are often located in the ‘supplementary material’, ‘acknowledgements’, ‘author notes’, ‘methods’ or ‘results’ sections. Search the article for the text ‘data available’ (to cover ‘data availability’ and ‘data available’). | ||
Data availability statement | Does the article state whether or not data are available? | Yes – the statement says that the data (or some of the data) are available / Yes – the statement says that the data are NOT available / No – there is no data availability statement / Other* |
Data sharing method | How does the statement indicate the data are available? | On request from the authors / personal or institution website / An online, third-party repository (eg, OSF, FigShare) / supplementary materials hosted by the journal / Other* |
Data accessibility | Can you access, download, and open the data files? | Yes / No / Other* |
Data documentation | Are the data files clearly documented? | Yes / No / Other* |
Data content | Do the data files appear to contain all of the raw data necessary to reproduce the reported findings? | Yes / No / Unclear / Other* |
Materials sharing | ||
Definitions: ‘Materials’ refers to any study items that would be needed to repeat the study, such as stimuli, survey instruments and computer code/software used for data collection, presentation stimuli or running experiments. | ||
Materials availability statement | Does the article state whether or not materials are available? | Yes – the statement says that the materials (or some of the materials) are available / Yes – the statement says that the materials are NOT available / No – there is no materials availability statement / Other* |
Materials sharing method | According to the statement, how are the materials accessible? | On request from the authors / personal or institution website / An online, third-party repository (eg, OSF, FigShare) / supplementary materials hosted by the journal / Other* |
Materials accessibility | Can you access, download and open the materials files? | Yes / No / Other* |
Analysis script sharing | ||
Definition: ‘Analysis scripts’ refers to specification of data preparation and analysis steps in the form of highly detail step-by-step instructions for using point-and-click software, analysis code (eg, R), or syntax (eg, from SPSS). | ||
Coder instructions: Check the article for an analysis script availability statement/link. They are often located in the ‘supplementary material’, ‘acknowledgements’, ‘author notes’, ‘methods’ or ‘results’ sections. Search for the text ‘analysis script’ and ‘analysis code’. | ||
Analysis script availability statement | Does the article state whether or not analysis scripts are available? | Yes – the statement says that the analysis scripts (or some of the analysis scripts) are available / Yes – the statement says that the analysis scripts are NOT available / No – there is no analysis script availability statement |
Analysis script sharing method | According to the statement, how are the analysis scripts accessible? | On request from the authors / Personal or institution website / An online, third-party repository (eg, OSF, FigShare) / supplementary materials hosted by the journal / Other* |
Analysis script accessibility | Can you access, download, and open the analysis script files? | Yes / No / Other* |
Funding | ||
Coder instructions: Funding is usually reported in a specific section, for example, ‘Author information’, or ‘Funding statement’. Search the article for the phrase ‘funding’. If you are unsure whether an organisation is an activity-related company, pharmaceutical company, other private company or public organisation, Google the organisation name and code accordingly. If it is unclear to you whether the funding is private or public, choose the ‘other’ option and enter ‘unclear’. | ||
Funding statement | Does the article include a statement indicating whether there were funding sources? | Yes – the statement says that there was funding from an activity-related company (eg, Nike, Fitbit) / Yes – funding from a pharmaceutical company (eg, Pfizer, GSK)/ Yes – funding from another private company (eg, Google, Coca Cola) / Yes – funding from a public organisation (eg, National Institute of Health Research)/ Yes – the statement says that there was no funding was provided / No – there is no funding statement / Unclear / Other* |
Conflict of interest | ||
Coder instructions: Conflicts of interest are usually reported in a specific section, for example, ‘Author information’ or ‘Conflict of interest statement’. Search the article for the phrases ‘conflict of interest’ and/or ‘competing interest’. If you are unsure whether an organisation is an activity-related company, pharmaceutical company, other private company or public organisation, Google the organisation name and code accordingly. If it is unclear to you whether the conflict of interest is private or public, choose the ‘other’ option and enter ‘unclear’. | ||
Conflict of interest statement | Does the article include a statement indicating whether there were any conflicts of interest? | Yes – the statement says that there was a conflict of interest from an activity-related company (eg, Nike, Fitbit) / Yes – conflict of interest from a pharmaceutical company (eg, Pfizer, GSK)/ / Yes – conflict of interest from another private company (eg, Google, Coca Cola) / Yes – conflict of interest from a public organisation (eg, National Institute of Health Research)/ Yes – the statement says that there is no conflict of interest / No – there is no conflict of interest statement / Other* |
Replication | ||
Definitions: ‘Replication’ refers to repetition of a previous study’s methods in order to ascertain whether similar findings can be obtained. | ||
Coder instructions: Search the abstract and introduction for the phrase ‘replicat’ (to cover ‘replication’, ‘replicates’ etc). Confirm the authors are using the phrase with the definition provided above. | ||
Replication statement | Does the article claim to report a replication study? | The article claims to report a replication study (or studies) / There is no clear statement that the article reports a replication study (or studies) / Other* |
Open access | ||
Coder instructions: To establish the open access status of the article: Go to https://openaccessbutton.org/ and enter the article’s doi (eg, ‘10.1371/journal.pcbi.1004574’) if available (if not, enter the article title). If a link is provided, check that you can access the article at the link. If the article is accessible answer ‘Yes’. If the article is not accessible at the provided link, or no link is provided, answer ‘No’. | ||
Open access status | Is the article open access? | Yes – found via open access button / Yes – found via other means / No – could not access article other than through paywall / Other |
*If a response marked with an asterisk is selected, the coder is asked to provide more detail in a free-text response box.