Unit researchers have joined colleagues at the University of Cambridge Faculty of Education and Bond University in Australia to call on scientific journals to amend their submission guidelines in a new commentary published today in Trials.  Their call comes after their analysis identified numerous research studies evaluating complex interventions that had been published with crucial information missing.

Missing details hamper research on schools-based interventions

An analysis published in 2022 by University of Cambridge researchers led by led by Mairead Ryan, a PhD student at the MRC Epidemiology Unit and Faculty of Education, reviewed reports from trials evaluating new school-based programmes to increase the amount of children’s physical exercise. The trial reports were published in 33 academic journals between 2015 and 2020 and collectively covered evaluations involving tens of thousands of pupils in hundreds of schools internationally.

They found that almost all of these reports left out key details about how teachers had been trained to deliver the interventions. Omitted details included where teachers had been trained to deliver the interventions, how that training had been provided, and whether it was adapted to meet the teachers’ needs and skills. Since each of these details are thought to influence the success of a trial, their analysis argues that the published reports are of limited scientific use.

Incomplete reporting also presents a challenge to anyone interested in implementing these interventions in other settings. Policy actors, practitioners and other researchers wishing to do so depend on studies being fully reported.

The case for TIDieR information

Several free checklists are available to researchers to ensure full reporting, so the researchers next examined whether reports from the trials of physical activity interventions for children and adolescents they has previously analysed had recorded the information stipulated in one such checklist: the Template for Intervention Description and Replication (or TIDieR). TIDieR was introduced in 2014, specifically for intervention research. It provides a minimum list of recordable items required to ensure that findings can be used and replicated. Their findings are published today in a commentary in the journal Trials

Mairead Ryan commented:

Inadequate intervention reporting is a widespread problem. If a trial is not described properly, scientists have no way of understanding the reasons behind its reported success or failure. This is preventing meaningful progress for researchers and practitioners.”

Her coauthors were Professor Tammy Hoffmann of Bond University in Australia, and from the University of Cambridge Professor Riikka Hofmann of the Faculty of Education and Dr Esther van Sluijs of the MRC Epidemiology Unit. Alongside the main reports, the team analysed related documents – such as protocols, process evaluations, and study websites – to see if these captured information specified in the TIDieR checklist.

Despite this thorough search, they found significant gaps in how the interventions were reported in 98% of cases.

Very basic information was often missing. For example, 62% of studies failed to specify where the teachers had been trained; the same proportion did not state whether the training had been adapted to meet teachers’ needs and skills, and 60% did not mention whether teachers were trained in a group or individually. As evidence suggests that each of these points can influence how successfully an intervention is delivered, their inclusion in a research report is essential.

When the research team assessed the submission guidelines of the academic journals that had published these reports, they found that just one encouraged the use of reporting checklists for all intervention components.

Dr Esther van Sluijs noted that:

Unfortunately, most of the journals we looked at did not encourage the use of reporting checklists for all intervention components in their submission guidelines. We would urge journals from all academic fields to review their guidance, to ensure a scientific evidence base that is fit for purpose.”

Academic journals respond

They then contacted each journal’s Editor-in-Chief (or equivalent) and asked them to update their guidelines. To date, 27 of the 33 have responded. Seven have updated their guidelines; others are still in discussion with their editorial teams and publishers.

One of the journals that acted in response is the British Journal of Sports Medicine. The journal’s Editor-in-Chief, Professor Jonathan Drezner, said:

Required reporting guidelines for intervention trials ensure that interventions are fully described and replicable. I am grateful to the researchers who advocated for a more rigorous and consistent reporting standard that ultimately raises the quality and scientific value of the clinical trials we publish.”

The authors hope more journals will follow suit. They point out that the 2013 Declaration of Helsinki, which lays out ethical principles for medical research involving human subjects, identifies full reporting as an “ethical obligation” for researchers, authors, editors and publishers.

Mairead Ryan believes that journal guidelines can play a major role in changing the status quo, saying:

We know that the quality of reporting is better in journals that endorse these checklists. Reporting guidelines are available for many study designs in over 100 languages on the EQUATOR Network website. Journals should encourage researchers to use them when submitting their work.”

Read more

• Ryan, M et al. Incomplete reporting of complex interventions: a call to action for journal editors to review their submission guidelines. Trials; 22 March 2023; DOI:10.1186/s13063-023-07215-1