To understand more about what works to reduce gambling harms
Well-designed and well-delivered evaluation will be a core part of the evidence base for widespread adoption of measures proven to reduce gambling harms.
Good evaluation looks not only at the process, but also the impact on people and behaviour. In order to achieve the overall aim of the strategy, a greater understanding is needed of the impact that interventions and activities have on how people gamble, how they experience harm, and how they respond to prevention and support activities and interventions. This means that evaluation needs to be built in from the beginning of an intervention or project.
The key principles of good evaluation, as set out in the existing protocol are:
- Robustness and credibility: Appropriate evaluation generates robust evidence. This includes drawing on quantitative and qualitative data and incorporating the consumer and/or user voice.
- Proportionality: Evaluation should be proportionate to the risk and scale of the intervention, so scale should be considered and documented at the outset.
- Independence: Independent evaluation is perceived as more objective, robust and credible, but may not be proportionate for all interventions.
- Transparency: Evaluations should be as open as possible about the rationale and details of the intervention, the evaluation process, results generated, and conclusions. Transparency increases confidence and credibility and allows stakeholders to think about how lessons learned can be transferred.
Evaluation has started to take place under the Commission’s independent research programme, to help understand the effectiveness of activities designed to reduce the risk of gambling harms, and working with partners to do so.
Examples include evaluating the impact of reducing friction for customers to encourage setting limits in online gambling, trialling safer gambling messaging on gaming machines in bingo premises, and the work to evaluate the effectiveness of multi-operator self-exclusion schemes.
To support evaluation across all partners
Evaluation is not confined to what the gambling industry does. It is important to find out what works in wider prevention programmes, and in treatment and support options. The success of the strategy will be reliant on how all parties involved approach and use evaluation to help determine the right mix of interventions and options to prioritise activity and reduce gambling harms. Good evaluation will enable and empower commissioning bodies to base funding and prioritisation decisions on evidence of what does and doesn’t work.
Longer term evaluation needs
Over the longer term, those involved in delivering the strategy may benefit from a more coordinated approach to evaluation. Options to prioritise and coordinate evaluation, including ownership and implementation, will be explored during the life of the strategy.
We will work with evaluation experts, and a range of stakeholders delivering interventions to reduce harm, including gambling businesses, to review and revise (PDF opens in new window) the 2016 RGSB evaluation protocol. It may be appropriate to further supplement the (PDF opens in new window) protocol and existing (PDF) guidance with additional or updated practical tools and advice, in order to raise standards in evaluation at the individual operator level and to begin to embed proportionate evaluation into both current practices, and at the beginning of new practices.
Crucially, we want to understand what does not work as much as what does, so that we can match our expectations to the most effective methods of reducing harms.
It is important that new measures or programmes are properly evaluated. For the Gambling Commission, this includes monitoring progress of significant new policy initiatives or regulatory changes and measuring progress against the strategy from the start. We will support and encourage other bodies to do the same.
Focus in Year 1
- Ensuring that interventions designed to reduce gambling harms are properly evaluated to understand the impact they have on how people gamble, how they experience harms and how they respond to prevention and support activities.
- Redesigning existing guidance to ensure they are fit for purpose and provide practical tools and advice to support evaluations across all stakeholders in order to raise standards and better understand what does and does not work.
- Continuing to progress existing evaluations on Multi-Operator Self-Exclusion Schemes and the extended trial of safer gambling messages on machines in Bingo and AGC premises.
- Sharing best practice as a result of evaluation in a transparent and coordinated manner through the National Strategy website.
- Consideration of a proposal from evaluation experts to support evaluation under the strategy, and the development of a new protocol.
Identified gaps being addressed
- Collating knowledge and learning from the range of evaluation activity being undertaken in a consistent and coordinated way which is available to all.
- Publication of a report by the Behavioural Insights Team, funded by GambleAware on trialling remote interventions into risky play (Year 1, Q2)
- Publication of the phase 1 report of the evaluation of Multi-Operator Self-Exclusion Schemes (Year 1, Q3)
- Develop and publish an evaluation framework for the Strategy (Year 1, Q4).
- Publish a revised evaluation protocol and resources for all Strategy partners to use (Year 1, Q4 - work to deliver this currently being scoped).
- Develop the framework for the evaluation of GambleAware-funded treatment and support services (Year 1, Q4)
- Publication of the National Institute for Health Research’s (NIHR) evaluation of policies and interventions for reducing gambling harms (Year 2).