MRC Epidemiology UnitWe sought to engage a variety of stakeholders (defined here ‘individuals and organisations that have a direct interest in the process and outcomes of a project, research or policy endeavour’) in the intervention prioritisation phase.
Given that no clear guidelines or consensus exist on how to do this, we utilsed a modified ‘Delphi’ approach that allowed us to engage with multiple, geographically spread, stakeholders (including young people) to prioritise physical activity promotion strategies in secondary school, two of which will be tested for feasibility and acceptability in a subsequent phase of the CASE program of research.
Methods
Step 1: Developing the list of potential intervention strategies
The prior phases of research within the CASE project (evidence reviews and secondary data analysis) enabled us to create an evidence-informed list of 30 candidate school environment-focused interventions to promote physical activity and reduce sedentary behaviour in secondary schools. Together with groups of students and teachers, and following discussions with academic experts, we identified 9 potential projects (see next section).
Step 2: Document preparation
Detailed documentation was prepared describing each proposed intervention strategy along with available evidence based information on the six pre-specified criteria for decision making (Health England, 2009). These include: reach, equality, acceptability, feasibility, effectiveness (for physical activity promotion, enjoyment of school, academic achievement, behaviour, mental health and well-being and teacher job satisfaction) and cost.
Two versions were developed for each proposed intervention – one for ‘adult’ stakeholders and one for ‘young people’. The PDF documents for each proposed intervention are available below (including the young person version).
- Active lessons (pdf) (young person version) (pdf)
- Activity permissive classrooms (pdf) (young person version) (pdf)
- Extending break time duration (pdf) (young person version) (pdf)
- Extending break time duration plus access to equipment (pdf) (young person version) (pdf)
- “K-a-day” plus outdoor trail (pdf) (young person version) (pdf)
- Physically active uniform (pdf) (young person version) (pdf)
- Playground equipment (pdf) (young person version) (pdf)
- Standing desks in classrooms (pdf) (young person version) (pdf)
- Teacher behaviours that support physical activity (pdf) (young person version) (pdf)
Methods of Delphi study
Participants
We recruited 37 people for the Delphi study, including (a) secondary school teachers, (b) secondary school students, (c) parents (of teenagers), (d) school governors, (e) local authority public health commissioners, (f) academics (public health and education-focused), (g) physical activity organisation or charity representatives, (h) Regional schools commissioners.
Delphi online tool development and pilot testing
An internally developed system facilitated the online Delphi process. Participants received a unique login ID via email. The online Delphi process had 2 rounds, separated by approximately 4 weeks. Each round took no more than 30 minutes. Participants could login to the online system to complete the Delphi exercises at a time and place convenient for them.
First scoring round
The online platform took participants through a series of questions (‘young person’ and ‘adult’ version (pdf), relating to the six decision-making criteria and asked participants to rank their top 3 intervention choices for each question in turn. On the final page, participants were asked to rank the interventions from the one they would most like to see implemented in a secondary school to their least favourite. Finally, we asked them to indicate the relative importance of each ‘effectiveness’ outcome (i.e., physical activity, mental health and well-being, academic achievement, enjoyment of school, concentration, behaviour and teacher satisfaction).
Lastly, participants were asked to provide suggestions for modifications to the proposed interventions, additional details to be included in the evidence summaries to be considered in Round 2, additional key prioritisation criteria and any additional effectiveness outcomes, and to provide overall comments. The free text comments from the online form were reviewed to identify key issues.
From the information provided in Round 1 data we developed group rankings for all questions as well as the perceived importance of the prioritisation criteria. These scores were calculated by weighting the respondents’ choices (i.e., 1st choice intervention = 9 points, 2nd choice intervention = 8 points etc.) and calculating a total score for each intervention.
Second scoring round
For round 2 of the Delphi (approximately 4 weeks later), all participants were emailed again with a reminder to login to the online system for a second time. In Round 2, participants received (by email) anonymous feedback on the findings from Round 1 relating to the rankings for each criteria and also the final overall rankings. Specifically, we created a document that was personalised for each participant. This document presented the participants’ own score (for each item) from round 1 as well as a summary of the top 3 choices for the whole sample. An example of the second round document is available here (pdf) (young person version available here) (pdf).
The participants were then asked to access the same online system already filled in with their original responses. This provided them with the option to change their original rankings, based on the anonymised group feedback.
Results
Most popular intervention
After Round 2, ‘active lessons’ was the most popular intervention (Figure 1), which was similar for all groups of participants (young people, educational professionals and public health professionals). The second and third most highly ranked interventions were ‘teacher behaviours that support physical activity’, followed by ‘activity permissive classrooms’.
Most important decision criterion
When asked what they considered the most important criterion to inform the ranking, the participants indicated that this was ‘effectiveness’ (see Figure 2). This was followed by ‘reach’ and ‘feasibility’.
Most important outcome for effectiveness
As these interventions can have an impact on different sorts of outcomes, we also asked participants to indicate which outcome they thought was most important for effectiveness. Participants consistently ranked ‘mental health and well-being’ as the most important outcome (see Figure 3). This was followed by ‘enjoyment of school’.
Conclusions
This project showed that it is feasible to engage with a diverse group of participants to efficiently and effectively prioritise public health interventions. The results showed a consistent preference for introducing ‘active lessons’ in secondary schools, an approach more frequently tested in primary schools. An active lessons teacher training programme is now being developed as part of the next phase of the CASE project, and we will be testing the implementation of this in 2017.
The results additionally show scope for a range of other strategies previously not tested or implemented in secondary schools. This includes extending the duration of break times while providing increased access to equipment and the introducing activity permissive classrooms. Although these will not be formally tested within CASE, we encourage schools and public health professionals to consider these strategies and report on their feasibility and effectiveness when trialled.
Finally, the results suggest that those looking to promote and deliver physical activity programs in secondary schools may do well to emphasise the potential benefits to mental health and wellbeing in their efforts to engage schools and students.
Further information
- Example participant feedback documents for Round 2 (i.e., final feedback) are available here:
- The results of the CASE prioritisation study have been published in BMJ Open: Morton KL, Atkin AJ, Corder K, Suhrcke M, Turner D, van Sluijs EM. Engaging stakeholders and target groups in prioritising a public health intervention: the Creating Active School Environments (CASE) online Delphi study. BMJ Open. 2017 Jan 13;7(1):e013340.