The Time of My Life Project: A Realist Evaluation

14 March 2017

Researchers:

Professor Sarah Galvani (Dept of Social Care and Social Work, Manchester Metropolitan University); Dr Denise Tanner (Dept of Social Policy and Social Work, University of Birmingham); Mr Stephen Jeffreys, Dr Beverley Smith, Dr Jo-Pei Tan, Ms Sophie Smailes, Dr Cherilyn Dance (Dept of Social Care and Social Work, MMU); Professor Kevin Albertson (Dept of Economics, MMU).

Key findings

  • TOML allows for the development of closer and more developed therapeutic relationships between users of the service and professionals. This appears to be key to its success.
  • Involvement of volunteers and peer supporters gives TOML a wider reach and breadth of support it otherwise could not offer
  • Group activities appear to be the most challenging element of TOML in terms of maximising attendance and success. However, those who attended valued them highly.
  • Working with an older client group required a change in attitudes and approach compared to ‘practice as usual’. It also required increased knowledge about health conditions and a commitment to partnership practice.
  • TOML would break-even providing people completing the programme maintain their target level of alcohol intake for 22 months (or 15 months if volunteer time is not included in the costs).
  • Three features of the TOML service were highlighted as most sustainable without further dedicated funding including the volunteer and peer supporter work, group work, partnership and training.

Background

The Time of My Life (TOML) project is a Birmingham-based alcohol service for people aged 50 years and older. It is one of a number of services delivered by the alcohol, drugs and gambling charity, Aquarius. This report presents findings from an evaluation conducted, primarily, in year 2 of the project. In particular, it presents the perspectives of a range of people who deliver or use TOML and should be read in conjunction with Aquarius’ TOML monitoring data.

There are few specialist alcohol services for older people in the UK (Wadd et al. 2011). Public Health Birmingham’s 2013 Drug and Alcohol Needs Assessment identified 21 agencies offering some level of support for people with substance problems (Kilgallon 2013). TOML is, however, the only known specialist older people’s alcohol service.

Methods

This evaluation used an adapted version of a realist evaluation framework (Pawson and Tilley, 2004). Realist evaluation applies three core concepts – Context, Mechanism, and Outcome – to determine what works, for whom, how and in what context.

Interviews and focus groups were carried out with key stakeholders (see Table 1) and thematically analysed. Quantitative training data were collected and analysed across different time points to determine the extent to which characteristics of service users, family/carers, profession and organisation were associated with experiences of drinking behaviour or lifestyle among service users and families. Further tests explored any changes in attitudes, skills and practices among a range of professionals during training and, subsequently, in practice. A break-even economic analysis was also carried out.

Table 1: Summary table of sample population and data collection approaches for TOML evaluation

Sample populationNumberData collection
Direct users of TOML services, that is, older people who had used or were currently using one or more of the services provided by TOML22Semi-structured Individual face to face or telephone interviews
Service users of TOML group activities15Focus groups
Family members who were receiving some form of support from TOML5Individual (semi-structured) or group interview
Volunteers and peer supporters who were helping to deliver one or more TOML services7Focus Group
Paid TOML staff, including managers, practitioners and support workers17Semi-structured Individual face to face or telephone interviews
Professionals/practitioners from other organisations who had received training from TOML337Paper based and online survey tool

Findings

TOML is a flexible and responsive partnership model, grounded in an understanding of the different treatment and support needs of older people. It offers a holistic approach which enables staff to respond to needs (e.g. social isolation, health and welfare support) that are beyond, but often related to, the alcohol intervention.

Volunteers and peer supporters were highly valued by their TOML colleagues. They allowed the TOML project to have a wide reach and breadth of support, and they offered life experience that many professionals could not or would not feel able to disclose. Through their involvement in TOML, volunteers and peer supporters were also able to develop their own skills and confidence.

TOML allows for the development of closer and more developed therapeutic relationships between service users and professionals. Service users report feeling supported, not being patronised, and gaining confidence to take control of their drinking. They also report improved physical and mental health, improved relationships with family and friends, and greater preparation for work. They felt strongly that ongoing support would be available to them from TOML or Aquarius should they need it.

Drink diaries were among the tools identified as helping people to change their drinking behaviour.

Location was key to ensuring service access. This applied to the availability of home visits but also to appropriate community venues for group work.

Group activities were the most challenging element of the TOML model in terms of attendance and success. Successful groups were highly valued by participants due to the peer support, socialisation, skills development and confidence building they offer. Groups also provided an alternative or distraction from drinking. Some groups focused on alcohol whereas others focused on social isolation and had little, if any, alcohol-specific content.

Participants felt the management and facilitation approach of groups was good, balancing encouragement and direction with a relaxed approach. Access difficulties were highlighted by both service users and volunteer and peer supporters. Provision of transport was identified as one way to improve attendance, as was increased promotion of the group activities by TOML staff to individual clients to ensure they were reaching socially isolated people.

Family members were accessed through relatives receiving TOML support, with some choosing to take up the opportunity of 1-1 support. Support for family members varied from providing information and education on alcohol to emotional support. Staff reported being able to offer more time to family members than would be possible in the parent service, Aquarius. Family groups were not run during this evaluation, which could suggest the challenges of group work extend to family member groups.

Working with an older client group required a change in attitudes and approach. Models of practice for mainstream services were not appropriate for people who often a) had complex needs as a result of age-related health conditions and b) had lived far longer with problematic alcohol use. Working with this client group required increased skills, particularly in relation to patience and listening skills. Staff working with a range of health issues also had to adapt their practice, often working with hospitals and other health professionals.

Key challenges include time pressures, staff resources, and working with new service providers. Although few service users identified areas for improvement, those who did sugegsted longer hours and greater flexibility in appointment times, more staff, and the avoidance of changes in staff. There was also was a need to improve referrals to the visiting service and to increase service availability.

Prior training in working with people with alcohol problems was low for non-substance specialists and previous training in working with older people with alcohol problems was low for both groups.

At the start of the training evaluation, substance specialists were better prepared than non-substance specialists to work with adults and with older people with alcohol problems. However, training increased the scores of both groups on this measure. A similar pattern was seen in relation to knowledge, sense of legitimacy and willingness to engage with alcohol issues. Among non-substance specialists, greater increases in attitude scores were seen for knowledge and legitimacy than for engagement (willingness or comfort with working with alcohol users). However, non-substance specialists felt more confident about being able to source role support after training. Current practice in working with older people with alcohol problems was low across both participant groups and there was little change in this for non-substance specialists by the end of the evaluation (data not available for substance specialists).

The total costs of TOML are approximately £495,141 per year (including volunteers’ time), or £340,040 (excluding volunteers’ time). The annual social savings are estimated to be £272,157.00. TOML would break-even providing people completing the programme maintain their target level of alcohol intake for 22 months (or 15 months if volunteer time is not included in the costs). However, these data are not available and there is a need for improved data collection in order to conduct a more definitive cost/benefit analysis.

The most sustainable features of the TOML service included: the volunteer and peer supporter work; group work; partnership; and training. The latter was seen more as a legacy of the project than a service that could continue without TOML. Staff felt older people had different needs and would not fit easily into a ‘standard model’ of service, necessitating the retention of a specialist older people alcohol service. Further development of existing services in the TOML model could include the TOML training, groups, volunteer and peer support programme and increased working with family members and carers. Increasing the number of staff was seen as key to developing the service.

Recommendations

  1. Disseminate TOML and develop it as an alternative model to engage and work with older people with alcohol problems and co-existing needs.
  2. Continue to commit resources to recruiting, training and retaining TOML volunteers and peer supporters.
  3. Review the continuation of groups at which there are no or few TOML clients and whose needs are not social isolation in addition to alcohol-related support.
  4. Consider options for shared transport arrangements or other travel support to maximise group attendance.
  5. Consider service provision out of ‘office hours’ to maximise support offered to family members who work.
  6. Review promotion of, and referrals to, the visiting service to ensure that service use is maximised.
  7. Formalise feedback routes to, and from, volunteers and peer supporters about their contribution and development needs.
  8. Continue training, but consider booster sessions or organisational support to ensure change in practice.
  9. Review the monitoring and recording of client data to ensure reliable analysis of unit consumption pre- and post- TOML service. Build in a follow-up period of up to 6-12 months post discharge to support effectiveness analysis.
  10. In future research, include an outcome measure that explores health and wellbeing.
  11. Carry out further research with a larger group of family members to determine their views on, and experiences of, the TOML service.
  12. Conduct a follow up survey to determine the progress of former services users after one, two and three years.
  13. Review data collection to ensure the possibility of a cost-effectiveness analysis in future

Further Information

Sarah Galvani is Professor of Adult Social Care at Manchester Metropolitan University. https://www2.mmu.ac.uk/social-care-and-social-work/staff/profile/ index.php?profile_id=1104

References

Department of Health (2016) UK Chief Medical Officers’ Alcohol Guidelines Review. Summary of the proposed new guidelines. London: Department of Health

Kilgallon, R. (2013) Drugs & Alcohol Needs Assessment 2013/14. Birmingham: Public Health Birmingham. Available online at: www.birminghampublichealth.co.uk/…/PHBNeedsAssessment2013-Final-part1(1).pdf [accessed 13 November 2016]

National Institute for Health and Care Excellence (NICE) (2015) Older people with social care needs and multiple long-term conditions. Available online at: nice.org.uk/guidance/ng22 [accessed 29 November 2016]

Office for National Statistics (2016) Statistical bulletin: Adult drinking habits in Great Britain: 2014. Available online at: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/drugusealcoholandsmoking/bulletins/opinionsandlifestylesurveyadultdrinkinghabitsingreatbritain/2014[accessed 26 November 2016]

Pawson, R. and Tilley, N. (2004) ‘Realist Evaluation’. Available online at: www.communitymatters.com.au/RE... [accessed 5 December 2016]

Wadd, S., Lapworth, K., Sullivan, M., Forrester, D. and Galvani, S. (2011) Working with older drinkers. Luton: University of Bedfordshire. Final report to funder.