Researching the effects of digital storytelling as a brief alcohol intervention for young people delivered in non-medical settings

Note: This report was funded and/or written by our predecessor organisation, the Alcohol Education Research Council (AERC).

This study documents important findings from an innovative alcohol intervention that was piloted in Caerphilly, South Wales. The intervention is innovative in terms of its use of media and its community-based setting in an area of high socio-economic deprivation where binge drinking among young people exceeds the national average (Communities That Care, CTC 2005). The intervention builds upon recommendations derived from young people in the local area who were interviewed during a previous study (Coleman and Cater 2007).

The intervention consists of two components, delivered over the course of a single 45 minute session. The first part, typically taking 20 minutes, involves viewing four, ‘Digital Stories’. Digital Stories are two to three minute long films created by young people. They consist of photographs and occasional background music which brings the script to life. The four stories used in this intervention address the consequences of binge drinking, through feeling ill, getting into trouble with the police, being in prison, and being hospitalised after a road traffic accident. Following the viewing of the Digital Stories, the intervention is continued in small groups (typically up to 30 young people) to discuss some of the issues raised. The facilitators of the intervention were a combination of staff employed in the community-based setting (e.g. a PSHE school teacher) and externally appointed staff (such as Youth and Social Workers). Further detail on Digital Stories can be viewed at: www.bbc.co.uk/tellinglives/.

The aim of the research was to assess the feasibility and effectiveness of this innovative media-based intervention seeking to reduce the prevalence of young people’s binge drinking in Caerphilly. The feasibility was assessed through:

• interviews with seven professionals involved with the project,
• four young people who created Digital Stories, and
• two focus groups with young people who participated in the intervention.

The effectiveness was established by assessing changes in young people’s drinking behaviour as well as the socio-psychological predictors of this behaviour, such as knowledge, attitudes, peer-group norms and intentions towards future drinking. Four community-based sites were approached and agreed to participate in the pilot study. These comprised two secondary schools and one youth group within Caerphilly County Borough who received the intervention, and one secondary school in neighbouring Gwent that acted as a control group.

Three self-report questionnaires were used to assess the effectiveness of the intervention. The first questionnaire was completed one-month prior to the intervention (T1), the second immediately after the intervention (T2) and the third six months after the intervention (T3). The samples from each site were based on a ‘take-all’ approach i.e. all those in attendance were invited to complete the questionnaires. This involved all attendees at the youth club irrespective of age, though at school sites this was exclusively amongst Year 10 (aged 14-15) students (for T1 and T2) who then progressed to Year 11 (aged 15-16) students at T3. The control group participants completed similar questionnaires at the same time-points as the intervention group. Finally, nine months after the intervention, 16 in-depth interviews were held with young people to explore their reactions and impacts of the intervention over the long-term. All qualitative data were thematically analysed, and the quantitative data were subject to a combination of descriptive and more inferential analyses, the latter merging individual responses across the three time-points.

The feasibility element of the study generated the following five main findings:

1. In this study, the Digital Story creators were requested to create stories that reflected their ‘lifestyle’. The pros and cons of defining the topic of interest to the Digital Story creators (alcohol in this case) must be considered. Advantages include the continual focus on the theme under study which therefore places less demand on the facilitator of the intervention to direct any subsequent discussion. Disadvantages include the possibility of less realistic stories being developed with young people trying to ‘please’ the facilitator.
2. Interviews with young people who created the stories showed they found the experience to be highly rewarding. Instances where the process had changed people’s own attitudes and behaviours towards alcohol were revealed.
3. The group discussion following the viewing of the Digital Stories was essential to draw focus to the objectives of the intervention. The group discussion component requires time and planning to enable a focussed and smooth delivery. A carefully planned and standardised group discussion guide is essential to ensure the consistency of delivery.
4. It is essential that the group discussion work should be limited to groups of 30 or fewer so that all participants have the opportunity to express their views and become actively involved in discussion.
5. Recommendations reported by the young people for future delivery of the Digital Stories were as follows:

• Make the Digital Stories longer (in this intervention they were a total of 5 minutes and 51 seconds), or show more of them.
• Consider making the stories more directly related to alcohol and the harms involved.
• Young people who viewed the intervention often felt that the stories were not realistic. This was because some young people perceived that the creators were reading from a script and assumed that they had been given this to read. By explaining the processes involved in developing Digital Stories to viewers, and emphasising that people are free to develop stories exactly how they wish, this problem could be avoided.
• Running the intervention in a separate session to a PSHE lesson may give it greater credibility.

To assess the effectiveness of the intervention, 1030 questionnaires were completed (423 at T1, 324 at T2 and 283 at T3). The sites (including the control) were similar in their socio-demographic profile with over 95% of young people reporting that they were ‘White British’, and all located in a relatively rural and socio-economically deprived area in South Wales.

The mean age recorded in the questionnaires increased from 14.7 years (T1) to 15.3 years (T3).

In tune with large scale survey data, the majority of young people had experience of drinking alcohol (typical 90%-95%) and around three-quarters of these had been drunk in the six months prior to the first round of questionnaires. The frequency of drunkenness over the previous week was generally similar among males and females, although males reported higher levels of consumption than females.

Due to a reduced number of questionnaires completed on the three occasions, the inferential analyses were performed by merging the T1 and T2 datasets, and furthermore by the T2 and T3 datasets. A total of 94 young people could be matched between T1 and T2, and 89 between T2 and T3. This was not considered problematic as drinking behaviour since the intervention was only recorded at T3. Therefore, T2 (as well as T1) acts as a baseline for recording behaviour prior to the intervention. Critically, both of the merged groups appeared to be representative of the wider dataset.

Knowledge levels surrounding alcohol and its effects were generally high (around 85-95% reporting correct answers to a nine-item question across all sites and time-points), thus making any changes in knowledge owing to the intervention difficult to detect. The notable exception for knowledge scores was the belief held by around one half of the sample (across all time-points) that, ‘Getting drunk once a week was not harmful’ (see Implications). Analyses from the merged dataset showed knowledge scored increased slightly for the intervention respondents, with the control respondents reporting a slight decrease. This was only evident between T1 and T2, and did not quite reach the level of statistical significance.

The control respondents reported an increase in positive attitudes to alcohol, as may be expected through greater experience associated with advanced age. This increase was not evident among the intervention group, suggesting that the intervention may be acting as a ‘protective effect’ in preventing expected increases in positive attitudes to alcohol. Further evidence of a ‘protective effect’ was seen in the response to young people’s intentions to get drunk in the future. From the ANOVA tests performed on the merged data, examination of the means showed that while control participants’ intentions were considerably higher at T2, indicating a greater intention (T1 Mean = 3.37; T2 Mean = 3.90), the intentions of the intervention participants remained similar at both time-points (T1 Mean = 3.26; T2 Mean = 3.29). This finding is particularly important since intention was found to be a significant predictor of drunkenness over the previous month, so impacts on intention are particularly crucial (see Implications).

As with the knowledge findings, these effects for attitudes and intentions were more evident between T1 and T2, indicating a short term impact of the intervention, which was not sustained six months later at T3. The in-depth interviews with the 16 young people nine months after the intervention confirmed this assertion about the minimal sustained impact. These interviews highlighted that the intervention had made young people think about alcohol, but primarily in the short-term.

The protective effect of the intervention was also evident in some of the descriptive statistics regarding drinking behaviour. For example, the proportion of young people who did not report drunkenness over the previous week and the previous month remained stable in the intervention group. To illustrate, the percentage not reporting drunkenness in the previous week for the intervention sites were stable (T1=53.3, T2=54.1, T3=52.7), while reductions were seen in the control group (T1=74.6, T2=53.8, T3=34.3) indicating a higher proportion of drunkenness in the control group.

The inferential statistics on the merged sample confirmed these changes in behaviour (between T2 and T3), although not quite at the level of statistical difference. Although there was no discernable difference in frequency of drunkenness in the last month, a two-factor ANOVA found that the intervention participants got drunk on fewer occasions in the last week (mean occasions last week = 1.57) compared to control participants (mean occasions last week = 2.00), with the difference approaching significance (F = 1.90, p =.07). This latter finding illustrates that the intervention has a degree of ability to predict drunkenness over the previous week and is arguably the most significant outcome of this evaluation.

Concluding points which are relevant to researchers, practitioners and policy-makers working in the health promotion field are presented below.

• Relative to other knowledge questions, the low scores in response to the statement, ‘Getting drunk once a week was not harmful’ was most interesting. This could be interpreted as the belief that getting drunk on one occasion per week is an acceptable and safe level of drinking. This may indicate that young people do not believe they are exceeding safe drinking limits (contrary to their levels of drinking reported), and thus furthers the importance of clarifying the definition of harmful drinking and increasing awareness of its consequences.

• Intentions towards future drunkenness was the most significant predictor of drinking over the previous month, when compared to gender, knowledge, attitudes, social norms, perceived control and perceived regret. The findings showed that intentions were shaped by attitudes towards drunkenness, perceived control, and regret. Therefore, focussing interventions on these three areas is likely to be beneficial when attempts are being made to change people’s drinking behaviour.

• For drinking behaviour in the previous week, perceived control over consumption (specifically, how easy it is to avoid drinking) was the most significant predictor. In the short term, therefore, it may be particularly beneficial to work of people’s control over their behaviour, perhaps by providing them with the confidence and skill to resist peer pressure.

• It should be remembered that changes in drinking behaviour may not necessarily be the only indicator of the intervention’s effectiveness. Particularly evident through the in-depth interviews held nine months after the intervention, and not possible to record through the questionnaires, the intervention was perceived as being useful in reaffirming the decisions of those young people who chose not to drink or who had limited experience of drunkenness. This impact should not be overlooked and implies that the effectiveness of the intervention may be increased if implemented among a younger age group where drinking was not prevalent.

Given some of the positive effects indicated by the results, consideration should be given to the development and repeated trial of this Digital Story intervention. Indications of the intervention’s potential to shape knowledge, attitudes, intentions and behaviour have been outlined above. Implementing the learning from the feasibility component would arguably improve the effectiveness of this intervention in the future.